Information processing apparatus, information processing method, and computer program

ABSTRACT

An information processing apparatus may include a control unit to control processing on an operation object, where an operation right of the operation object is given to a user, displayed on a display unit based on detection of a user from at least one information signal input to the apparatus from outside of the apparatus.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority from Japanese Patent ApplicationJP 2012-005258 filed in the Japan Patent Office on Jan. 13, 2012, theentire contents of which is hereby incorporated by reference.

TECHNICAL FIELD

The technique disclosed in this specification relates to an informationprocessing apparatus provided with a display screen which is also usedas an input unit based on a touch panel scheme or the like, aninformation processing method, and a computer program, and particularlyto an information processing apparatus, which includes a large screenand is commonly used by a plurality of users, on which the users performcooperative work through operations on the touch panel, an informationprocessing method, and a computer program.

BACKGROUND ART

Recently, a tablet terminal provided with a display screen which is alsoused as an input unit based on a touch panel scheme or the like has beenrapidly distributed. Since the tablet terminal has a widget or a desktopas an interface, and it is easy to visually understand an operationmethod, users can readily use the tablet terminal as compared with apersonal computer on which an input operation is performed via akeyboard or a mouse.

For example, a touch sensitive device which reads data belonging to atouch input relating to a multi-point sensing device from themulti-point sensing device such as a multi-point touch screen andidentifies a multi-point gesture based on the data from the multi-pointsensing device has been proposed (see Patent Literature 1, for example).

Generally, multiple operation target objects to be operated by a userare arranged in various orientations on a screen of the tablet terminal.The individual operation target objects are reproduced content such as avideo image and a stationary image, e-mails and messages received fromother users, and the like. It is necessary for the user to individuallyperform a rotation operation on a tablet terminal main body in order todisplay a desired operation target object so as to face the user. In thecase of a tablet terminal with an A4 or A5 size, it is easy to performthe rotation operation. In the case of a large screen of about severaltens of inches, however, performing the rotation operation on the tabletterminal every time a single user operates the operation target objectis a nuisance for the user.

In addition, a use state in which a plurality of users individuallyperform operations on operation target objects in a tablet terminal witha large screen at the same time can be also considered.

For example, a tablet terminal which identifies an area between a rightarm and a left arm when a proximity sensor detects a position, at whicha user is present, at a side edge of the terminal and performs mappingin a touch point area of the user has been proposed (see Non PatentLiterature 1, for example). The tablet terminal can be designed suchthat another user cannot perform certain operations such as an operationfor rotating an operation target object, which is being operated by acertain user, in a direction in which the operation target object facesthe user themselves by setting an operation right of each user for eachoperation target object or inhibiting additional user participation inadvance when the tablet terminal detects a plurality of users.

As the use state in which a plurality of users commonly use the tabletterminal with the large screen, however, there may also be a case inwhich the users exchange operation target objects to perform cooperativework as well as the aforementioned case in which the users individuallyperform operations on the operation target objects. It is difficult torealize cooperative work if a touch point area occupied by each user isset and each user performs operations on the operation target objects,operation rights for which have been provided, in the individual area.

In addition, if a GUI displayed on the screen of the terminal isconstant regardless of a distance from a user to the screen or a userstate, there is a problem in that the size of information shown on thescreen is excessively small and cannot be well understood by a user whois located at a distant position, or a problem in that an amount ofinformation shown on the screen is small even though the user ispositioned at a nearby position. Similarly, if input means for allowingthe user to operate the terminal is constant regardless of the distancefrom the user to the screen or the user state, the user cannot performany operations since there is no remote controller even though the useris positioned at a nearby position to the terminal, or the user isrequired to approach the terminal in order to operate the touch panel,which is inconvenient.

In addition, according to a conventional subject display system, animage of an actual object is displayed on the screen withoutconsideration of real size information thereof. For this reason, thereis a problem in that the size of the object displayed varies inaccordance with a size or resolution (dpi) of the screen.

In addition, when movie content of a plurality of sources issimultaneously displayed on the screen in a display system in a parallelmanner or in a superimposed manner, a magnitude relationship between thesimultaneously displayed images is not correctly displayed, sizes andpositions of corresponding areas for the images are not adjusted, andtherefore, an image which cannot be easily viewed by the user isobtained.

In addition, since the user cannot easily view the screen if a directionof the screen of the terminal or the like which is provided with arotation mechanism is changed, it is necessary to rotate the displayscreen.

CITATION LIST Patent Literature

-   PTL 1: Japanese Unexamined Patent Application Publication No.    2010-170573

Non Patent Literature

-   NPL 1: http://www.autodeskresearch.com/publications/medusa (as of    Dec. 15, 2011)

SUMMARY OF INVENTION Technical Problem

An object of the technique disclosed in this specification is to providean excellent information processing apparatus which includes a largescreen and is commonly used by a plurality of users, on which the userscan preferably perform cooperative work through operations on the touchpanel, an information processing method, and a computer program.

In addition, an object of the technique disclosed in this specificationis to provide an excellent information processing apparatus which isconstantly and conveniently operated by a user regardless of a userposition or a user state; an information processing method, and acomputer program.

In addition, an object of the technique disclosed in this specificationis to provide an excellent information processing apparatus which canconstantly display an image of an object in an appropriate size on thescreen regardless of a size of an actual object and or a size orresolution of the screen, an information processing method, and acomputer program.

In addition, an object of the technique disclosed in this specificationis to provide an excellent information processing apparatus which canpreferably display movie content of a plurality of sources at the sametime on the screen in a parallel manner or in a superimposed manner, aninformation processing method, and a computer program.

In addition, an object of the technique disclosed in this specificationis to provide an excellent information processing apparatus which canoptimally adjust a display mode of movie content at an arbitraryrotation angle in a shifting process thereof when a main body isrotated, an information processing method, and a computer program.

Solution to Problem

The present application was made in consideration of the above problems.

According to an embodiment of the present disclosure, there is providedan information processing apparatus that may include a control unit tocontrol processing on an operation object, where an operation right ofthe operation object is given to a user, displayed on a display unitbased on detection of a user from at least one information signal inputto the apparatus from outside of the apparatus.

According to an embodiment of the present disclosure, there is providedan information processing method that may include controlling, by aprocessor, of processing on an operation object, where an operationright of the operation object is given to a user, displayed on a displayunit based on detection of a user from at least one information signalinput to an information processing apparatus from outside of theapparatus.

According to an embodiment of the present disclosure, there is provideda non-transitory recording medium recorded with a program executable bya computer, where the program may include controlling processing on anoperation object, where an operation right of the operation object isgiven to a user, displayed on a display unit based on detection of auser from at least one information signal input to an informationprocessing apparatus from outside of the apparatus.

Advantageous Effects of Invention

According to the technique disclosed in this specification, it ispossible to provide an excellent information processing apparatus whichincludes a large screen and is commonly used by a plurality of users, onwhich the users can preferably perform cooperative work throughoperations on the touch panel, an information processing method, and acomputer program.

In addition, according to the technique disclosed in this specification,it is possible to provide an excellent information processing apparatuswhich is conveniently used by a user by optimizing the display GUI orthe input means in accordance with a user position or a user state, aninformation processing method, and a computer program.

In addition, according to the technique disclosed in this specification,it is possible to provide an excellent information processing apparatuswhich can constantly display an image of an object at the appropriatesize on the screen regardless of a size of an actual object and or asize or resolution of the screen, an information processing method, anda computer program.

In addition, according to the technique disclosed in this specification,it is possible to provide an excellent information processing apparatuswhich can show the user an easily viewable screen with adjusted sizesand positions of corresponding areas for the images by performingnormalization processing on the images when movie content of a pluralityof sources is simultaneously displayed on the screen in the parallelmanner or in the superimposed manner, an information processing method,and a computer program.

In addition, according to the technique disclosed in this specification,it is possible to provide an excellent information processing apparatuswhich can optimally adjust a display mode of movie content at anarbitrary rotation angle in a shifting process thereof when a main bodyis rotated, an information processing method, and a computer program.

Other purposes, features, and advantages of the technique disclosed inthis specification will become apparent by detailed description based onembodiments and accompanying drawing which will be described later.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram showing an example (Wall) of a use state of aninformation processing apparatus 100 provided with a large screen.

FIG. 2 is a diagram showing another example (Tabletop) of a use state ofthe information processing apparatus 100 provided with the large screen.

FIG. 3A is a diagram showing another example of a use state of theinformation processing apparatus 100 provided with the large screen.

FIG. 3B is a diagram showing another example of a use state of theinformation processing apparatus 100 provided with the large screen.

FIG. 3C is a diagram showing another example of a use state of theinformation processing apparatus 100 provided with the large screen.

FIG. 4 is a diagram schematically showing a functional configuration ofthe information processing apparatus 100.

FIG. 5 is a diagram showing an internal configuration of the inputinterface unit 110.

FIG. 6 is a diagram showing an internal configuration of an outputinterface unit 130.

FIG. 7 is a diagram showing an internal configuration for a computationunit 120 performing processing on an operation target object.

FIG. 8 is a diagram showing a state in which a user occupying area isset in the screen.

FIG. 9A is a diagram showing a state in which operation target objects#1 to #6 are randomly oriented before a user occupying area A is set.

FIG. 9B is a diagram showing a state in which directions of theoperation target objects #1 to #6 are shifted to a direction in whichthe operation target objects #1 to #6 face a user A by setting the useroccupying area A for the user A.

FIG. 10 is a diagram showing a state in which a user occupying area Bfor a user B and a common area are additionally set in the screen sinceit has been detected that the user B is present as well as the user A.

FIG. 11 is a diagram showing a state in which a user occupying area Dfor the user B and the common area are additionally set in the screensince it has been detected that a user D is present as well as the usersA and B.

FIG. 12 is a diagram showing a state in which a user occupying area Cfor the user B and the common area are additionally set in the screensince it has been detected that a user C is present as well as the usersA, B, and D.

FIG. 13A is a diagram showing an example of an area division pattern inwhich the screen is divided into the user occupying area for each userin accordance with a shape and a size of the screen and a number of theusers.

FIG. 13B is a diagram showing an example of an area division pattern inwhich the screen is divided into the user occupying area for each userin accordance with a shape and a size of the screen and a number of theusers.

FIG. 13C is a diagram showing an example of an area division pattern inwhich the screen is divided into the user occupying area for each userin accordance with a shape and a size of the screen and a number of theusers.

FIG. 13D is a diagram showing an example of an area division pattern inwhich the screen is divided into the user occupying area for each userin accordance with a shape and a size of the screen and a number of theusers.

FIG. 13E is a diagram showing an example of an area division pattern inwhich the screen is divided into the user occupying area for each userin accordance with a shape and a size of the screen and a number of theusers.

FIG. 14 is a flowchart showing a processing procedure for monitor areadivision executed by a monitor area dividing unit 710.

FIG. 15 is a diagram showing a state in which an operation target objectis automatically rotated in a direction in which the operation targetobject faces the user by performing dragging or throw displacement ofthe operation target object to a user occupying area.

FIG. 16A is a diagram showing a state in which an operation targetobject newly appearing in a user occupying area is automatically rotatedin a direction in which the operation target object faces a user.

FIG. 16B is a diagram showing a state in which an operation targetobject newly appearing in a user occupying area is automatically rotatedin a direction in which the operation target object faces a user.

FIG. 17 is a flowchart showing a procedure optimization processing foran operation target object executed by an object optimal processing unit720.

FIG. 18 is a diagram showing a state in which a rotation direction iscontrolled in accordance with a position at which the user performs atouch operation on an operation target object.

FIG. 19 is a diagram showing a state in which the rotation direction iscontrolled in accordance with a position at which the user performs atouch operation on an operation target object.

FIG. 20 is a diagram showing an example of an interaction for exchangingan operation target object between the information processing apparatus100 and a terminal owned by a user.

FIG. 21 is a flowchart showing a processing procedure for equipmentcoordination data exchange executed by an equipment coordination dataexchanging unit 730.

FIG. 22 is a diagram showing a state in which an operation target objectis displaced between user occupying areas to copy the operation targetobject.

FIG. 23 is a diagram showing an internal configuration for thecomputation unit 120 performing optimization processing in accordancewith a user distance.

FIG. 24A is a diagram showing, in a table, GUI display optimizationprocessing by a display GUI optimization unit 2310 in accordance with auser position and a user state.

FIG. 24B is a diagram showing screen shift of the information processingapparatus 100 in accordance with a user position and a user state.

FIG. 24C is a diagram showing screen shift of the information processingapparatus 100 in accordance with a user position and a user state.

FIG. 24D is a diagram showing screen shift of the information processingapparatus 100 in accordance with a user position and a user state.

FIG. 24E is a diagram showing screen shift of the information processingapparatus 100 in accordance with a user position and a user state.

FIG. 25A is a diagram showing a display example of a screen for randomlydisplaying and automatically zapping various operation target objects.

FIG. 25B is a diagram showing a display example of a screen in whichdisplay positions and sizes of a plurality of operation target objectsto be automatically zapped are momentarily changed.

FIG. 26 is a diagram showing a screen display example in a state inwhich a user does not perform any operations while watching atelevision.

FIG. 27A is a diagram showing a screen display example in a state inwhich the user is performing an operation on the television.

FIG. 27B is a diagram showing a screen display example in which the useris performing an operation on the television.

FIG. 28 is a diagram showing, in a table, input means optimizationprocessing by an input means optimization unit 2320 in accordance with auser position and a user state.

FIG. 29 is a diagram showing, in a table, distance detection schemeshifting processing by a distance detecting scheme shifting unit 2330 inaccordance with a user position.

FIG. 30 is a diagram illustrating a problem in a conventional subjectdisplay system.

FIG. 31 is a diagram illustrating a problem in a conventional subjectdisplay system.

FIG. 32 is a diagram showing an internal configuration for thecomputation unit 120 performing object real size display processing inaccordance with monitor performance.

FIG. 33 is a diagram showing an example in which an image of a sameobject is displayed at an actual size on screens with different monitorspecifications.

FIG. 34 is a diagram showing an example in which images of two objectswith different real sizes are displayed on a same screen while a mutualmagnitude relationship is correctly maintained.

FIG. 35 is a diagram showing an example in which an image of an objectis displayed at real size.

FIG. 36 is a diagram showing an example in which an image of an objectdisplayed at real size is rotated or a posture thereof is converted.

FIG. 37A is a diagram inviting a state in which target real sizeinformation is estimated.

FIG. 37B is a diagram showing a state in which real size displayprocessing for an operation target object is performed based on theestimated target real size information.

FIG. 38A is a diagram showing a state in which sizes and positions offaces of users who are performing a video chat are not adjusted.

FIG. 38B is a diagram showing a state in which the sizes and thepositions of the faces of the users who are performing video chattingare adjusted by normalization processing among a plurality of images.

FIG. 39A is a diagram showing a state in which sizes and positions offigures of a user and an instructor which are displayed on the screen ina parallel manner are not adjusted.

FIG. 39B is a diagram showing a state in which the sizes and thepositions of the figures of the user and the instructor which aredisplayed on the screen in the parallel manner are adjusted bynormalization processing among the plurality of images.

FIG. 39C is a diagram showing a state in which the figures of the userand the instructor which have been normalized by the normalizationprocessing among the plurality of images are displayed in a superimposedmanner.

FIG. 40A is a diagram showing a state in which a product sample image isnot superimposed at an appropriate position in a correct magnituderelationship with a movie of a user themselves.

FIG. 40B is a diagram showing a state in which the product sample imageis displayed in a superimposed manner at the appropriate position in thecorrect relationship with the movie of the user themselves bynormalization processing among a plurality of images.

FIG. 41 is a diagram showing an internal configuration for thecomputation unit 120 performing image normalization processing.

FIG. 42 is a diagram showing a display mode in which an entire area ofmovie content is displayed such that the movie content is completely inthe screen at an arbitrary rotation angle.

FIG. 43 is a diagram showing a display mode in which a focused area inthe movie content is maximized at each rotation angle.

FIG. 44 is a diagram showing a display mode in which the movie contentis rotated such that an invalid area does not appear.

FIG. 45 is a diagram showing a relationship of a zooming rate of themovie content with respect to a rotation position in the display modesshown in FIGS. 42 to 44.

FIG. 46 is a flowchart showing a processing procedure for controlling adisplay mode of movie content by the computation unit 120 when theinformation processing apparatus 100 is rotated.

FIG. 47 is a diagram showing an internal configuration for thecomputation unit 120 performing processing for adjusting a display modeof movie content at an arbitrary rotation angle of a main body of theinformation processing apparatus 100 in a shifting process thereof.

DESCRIPTION OF EMBODIMENTS

Hereinafter, detailed description will be given of embodiments of thetechnique disclosed in this specification with reference to thedrawings.

A. System Configuration

An information processing apparatus 100 according to the embodiment isprovided with a large screen, and as main use states, there may be a“Wall” for hanging the information processing apparatus 100 on the wallas shown in FIG. 1 and a “Tabletop” for installing the informationprocessing apparatus 100 on a table as shown in FIG. 2.

In the “Wall” state shown in FIG. 1, the information processingapparatus 100 is attached in a rotatable and detachable manner on thewall by a rotation and attachment mechanism unit 180, for example. Inaddition, the rotation and attachment mechanism unit 180 is also used asan electrical contact between the information processing apparatus 100and the outside, a power cable and a network cable (both of which arenot shown in the drawing) are connected to the information processingapparatus 100 through the rotation and attachment mechanism unit 180,and the information processing apparatus 100 can receive drive powerfrom commercial AC power and access various servers on the Internet.

As will be described later, the information processing apparatus 100 isprovided with a distance sensor, a proximity sensor, and a touch sensorand can grasp a position (distance, orientation) of a user which facesthe screen. When the user has been detected, or in a state in which theuser is being detected, it is possible to express a wave-like detectionindicator (which will be described later) and perform illuminationexpression representing a detection state on the screen to give the uservisual feedback.

The information processing apparatus 100 is designed to automaticallyselect an optimal interaction in accordance with a user position. Forexample, the information processing apparatus 100 automatically selectsor adjusts a GUI (Graphical User Interface) display of frameworks ofoperation target objects, information density, and the like inaccordance with the user position. In addition, the informationprocessing apparatus 100 can automatically select one of a plurality ofinput means such as touching the screen, proximity, a gesture using ahand or the like, a remote controller, an indirect operation by a userstate, and the like in accordance with a user position or a distance tothe user.

In addition, the information processing apparatus 100 is also providedwith one or more cameras and can recognize not only the user positionbut also a person, a subject, or a device from an image captured by thecameras. Moreover, the information processing apparatus 100 is alsoprovided with a very-short-range communication unit and can directly andnaturally exchange data with a device owned by a user who approaches aposition at a very short range.

Operation target objects as targets to be operated by the user aredefined on the large screen in the Wall state. The operation targetobjects have specific display areas for arbitrary internet sites,applications, functional modules such as widgets as well as videoimages, stationary images, and text contents. In the operation targetobjects, content received from television broadcasts, content reproducedfrom a recording medium, streaming video images obtained by a network,video images obtained by another device such as a mobile terminal ownedby the user, stationary image content, and the like are included.

When the rotation position of the information processing apparatus 100hung on the wall is set such that the large screen is laterallyinstalled as shown in FIG. 1, it is possible to display a movie as anoperation target object corresponding to an entire screen size while aworld view depicted in the movie is maintained substantially as it is.

Here, when the rotation position of the information processing apparatus100 hung on the wall is set such that the large screen is in thelongitudinal direction, it is possible to arrange three screens with anaspect ratio of 16:9 in the vertical direction as shown in FIG. 3A. Forexample, three kinds of content #1 to #3 such as broadcast contentsimultaneously received from different broadcasting stations, contentreproduced from a recording medium, streaming video images on a network,or the like can be arranged in the vertical direction and simultaneouslydisplayed. Moreover, when the user performs an operation with a tip of afinger, for example, in the vertical direction on the screen, thecontent is scrolled in the vertical direction as shown in FIG. 3B. Inaddition, when the user performs an operation with the tip of the fingerin the horizontal direction at any position in the three stages, thescreen in the stage is scrolled in the horizontal direction as shown inFIG. 3C.

On the other hand, in the “Tabletop” state shown in FIG. 2, theinformation processing apparatus 100 is directly installed on the table.While the rotation and attachment mechanism unit 180 is also used as anelectrical contact in the use state shown in FIG. 1 (as describedabove), the electrical contact to the information processing apparatus100 is not found in the state in which the information processingapparatus 100 is installed on the table as shown in FIG. 2. Thus, theinformation processing apparatus 100 may be configured to be operable bya built-in battery with no power source in the Tabletop state shown inthe drawing. In addition, if the information processing apparatus 100 isprovided with a wireless communication unit corresponding to a mobilestation function of a wireless LAN (Local Area Network), for example,and the rotation and attachment mechanism unit 180 is provided with awireless communication unit corresponding to an access point function ofthe wireless LAN, the information processing apparatus 100 can accessvarious servers on the Internet through wireless communication with therotation and attachment mechanism unit 180 as an access point even inthe Tabletop state.

A plurality of operation target objects as targets to be operated aredefined on the screen of the large screen on the Tabletop. The operationobjects have specific display areas for arbitrary internet sites,applications, and functional modules such as widgets as well as videoimages, stationary images, text content.

The information processing apparatus 100 is provided with a proximitysensor for detecting a presence or a state of users at each of four sideedge portions of the large screen. In the same manner as describedabove, a user who has approached the large screen may be imaged by thecamera and person recognition may be performed. In addition, thevery-short-range communication unit detects whether or not the user, whohas been detected to be present, owns a device such as a mobile terminaland detects a request for exchanging data from another terminal owned bythe user. When the user or the terminal owned by the user has beendetected, or in a state in which the user is being detected, it ispossible to express a wave-like detection indicator or performillumination expression (which will be described later) representing adetection state on the screen to give the user visual feedback.

When presence of a user is detected by the proximity sensor or the like,the information processing apparatus 100 uses the detection result forUI control. It is possible to use the detection result for more detailedUI control if not only whether or not the user is present but alsopositions of a trunk, both arms and legs, and a head are detected. Inaddition, the information processing apparatus 100 is also provided withthe very-short-range communication unit and can directly and naturallyexchange data with a device owned by a user who approaches a position ina very short range (same as above).

Here, as an example of the UI control, the information processingapparatus 100 sets a user occupying area for each user and a common areawhich is commonly used by the users in the large screen in accordancewith the detected user arrangement. Then, a touch sensor input of eachuser in the user occupying area and the common area is detected. A shapeof the screen and an area dividing pattern are not limited to rectanglesand can be adapted to a square shape, a circular shape, or a shapeincluding a solid such as a circular cone.

If the screen of the information processing apparatus 100 is enlarged,room for spaces on which a plurality of users simultaneously performtouch inputs is generated in the Tabletop state. As described above, itis possible to realize comfortable and efficient simultaneous operationsby the plurality of users by setting the user occupying area for eachuser and the common area in the screen.

An operation right of an operation target object placed in a useroccupying area is given to a corresponding user. The operation right ofthe operation target object is moved to the user themselves by movingthe operation target object from the common area or a user occupyingarea for another user to their own user occupying area. Moreover, if theoperation target object enters to their own user occupying area, displayis automatically changed such that the operation target object facesthemselves.

When the operation target object is displaced between the user occupyingareas, the operation target object is physically displaced in a naturaloperation in accordance with a touch position on which a displacementoperation is performed. In addition, an operation for dividing orcopying one operation target object can be performed by mutually pullingthe operation target object by users.

FIG. 4 schematically shows a functional configuration of the informationprocessing apparatus 100. The information processing apparatus 100 isprovided with an input interface unit 110 to which an information signalfrom the outside is input, a computation unit 120 which performscomputation processing for control of the display screen based on theinput information signal, an output interface unit 130 which outputsinformation to the outside based on the computation result, ahigh-capacity storage unit 140 configured by a hard disk drive (HDD) orthe like, a communication unit 150 which is connected to an externalnetwork, a power unit 160 which deals with drive power, and a televisiontuner unit 170. The storage unit 140 stores thereon various processingalgorithms executed by the computation unit 120 and various kinds ofdatabase used in computation processing by the computation unit 120.

Main functions of the input interface unit 110 include detection of userpresence, detection of a touch operation by the detected user on thescreen, namely a touch panel, detection of a device such as a mobileterminal owned by the user, and processing for receiving transmitteddata from the device. FIG. 5 shows an internal configuration of theinput interface unit 110.

The remote control receiving unit 501 receives a remote control signalfrom a remote controller or a mobile terminal. A signal analysis unit502 performs demodulation and decoding processing on the received remotecontrol signal and obtains a remote control command.

As a camera unit 503, a monocular type or one or both of a two-lens typeand an active type is employed. The camera unit 503 is configured by animaging element such as a CMOS (Complementary Metal Oxide Semiconductor)or a CCD (Charge Coupled Device). In addition, the camera unit 503 isprovided with a camera control unit for panning, tilting, zooming, andthe like. The camera unit 503 informs the computation unit 120 of camerainformation relating to panning, tilting, zooming, and the like andcontrol panning, tilting, and panning, tilting, and zooming of thecamera unit 503 can be controlled in accordance with camera controlinformation from the computation unit 120.

An image recognition unit 504 performs recognition processing on animage captured by the camera unit 503. Specifically, a user face or ahand motion is detected by background differencing, a user face includedin a captured image or a human body is recognized, and a distance to auser is recognized.

A microphone unit 505 performs audio input of sound or conversationgenerated by users. An audio recognition unit 506 performs audiorecognition on the input audio signal.

A distance sensor 507 is configured by a PSD (Position SensitiveDetector), for example, and detects a signal replied from a user oranother subject. A signal analysis unit 508 analyzes the detectionsignal and measures a distance to the user or the substance. Apyroelectric sensor or a simple camera can be used as the distancesensor 507 as well as the PDS sensor. The distance sensor 507 constantlymonitors whether or not a user is present within a radius of 5 to 10meters, for example, from the information processing apparatus 100. Forthis reason, it is preferable that a low power consumption sensorelement be used for the distance sensor 507.

A touch detection unit 509 is configured by a touch sensor superimposedon the screen and outputs a detection signal from a position, which atip of a user's finger touches, on the screen. A signal analysis unit510 analyzes the detection signal and obtains position information.

A proximity sensor 511 is installed at each of four side edge portionsof the large screen to detect that a user body has approached the screenbased on an electrostatic capacitance scheme. A signal analysis unit 512analyzes the detection signal.

A very-short-range communication unit 513 receives a non-contactcommunication signal from a device or the like owned through a user byNFC (Near Field Communication). A signal analysis unit 514 performsdemodulation and decoding processing on the received signal to obtainreceived data.

A triaxial sensor unit 515 is configured by a gyro and detects a postureof the information processing apparatus 100 about each of x, y, and zaxes. A GPS (Global Positioning System) receiving unit 516 receives asignal from a GPS satellite. A signal analysis unit 517 analyzes signalsfrom the triaxial sensor unit 515 and the GPS receiving unit 516 toobtain position information and posture information of the informationprocessing apparatus 100.

An input interface unifying unit 520 unifies inputs of the aboveinformation signals and passes the unified input to the computation unit120. In addition, the input interface unifying unit 520 unifies analysisresults of the signal analysis units 508, 510, 512, and 514, obtainsposition information of the users who are present in the surroundings ofthe information processing apparatus 100, and passes the positioninformation to the computation unit 120.

Main functions of the computation unit 120 includes computationprocessing such as UI screen generation processing based on a userdetection result by the input interface unit 110, a screen touchdetection result, and data received from the device owned by the userand outputs of the computation result to the output interface unit 130.The computation unit 120 can realize computation processing for eachapplication by loading and executing an application program installed onthe storage unit 140, for example. A functional configuration of thecomputation unit 120 corresponding to each application will be describedlater.

Main functions of the output interface unit 130 includes UI display onthe screen based on the computation result by the computation unit 120and transmission of data to the device owned by the user. FIG. 6 showsan internal configuration of the output interface unit 130.

An output interface unifying unit 610 collectively deals withinformation output based on computation results of monitor divisionprocessing by the computation unit 120, object optimal processing,equipment coordination data exchange processing, and the like.

The output interface unifying unit 610 instructs a content display unit601 to output an image and audio to a display unit 603 and a speakerunit 604 for video image or stationary image content such as contentreceived from television broadcasts, content reproduced from a recordingmedium such as a Blu-ray disc.

In addition, the output interface unifying unit 610 instructs a GUIdisplay unit 602 to display a GUI of an operation target object and thelike on the display unit 603.

In addition, the output interface unifying unit 610 instructs anillumination display unit 605 to output and display illumination, whichrepresents a detection state, from an illumination unit 606.

In addition, the output interface unifying unit 610 instructs thevery-short-range communication unit 513 to transmit data throughnon-contact communication to the device owned by the user.

The information processing apparatus 100 can detect a user based onrecognition of an image captured by the camera unit 503 and detectionsignals by the distance sensor 507, the touch detection unit 509, theproximity sensor 511, the very-short-range communication unit 513, andthe like. In addition, it is possible to identify who the detected useris by face recognition of the image captured by the camera unit 503 andby causing the very-short-range communication unit 513 to recognize thedevice owned by the user. The identified user can log in the informationprocessing apparatus 100. It is a matter of course that accounts whichcan be logged in can be limited to specific users. In addition, theinformation processing apparatus 100 can separately use the distancesensor 507, the touch detection unit 509, and the proximity sensor 511depending on a user position and a user state and receive an operationfrom the user.

In addition, the information processing apparatus 100 is connected to anexternal network through the communication unit 150. The connectionstate to the external network may be wired or wireless connection. Theinformation processing apparatus 100 can communicate with other devicessuch a mobile terminal including a smartphone, a tablet terminal, andthe like owned by the user through the communication unit 150. Aso-called “three screens” can be configured by a combination of threetypes of apparatuses, namely the information processing apparatus 100,the mobile terminal, and the tablet terminal. The information processingapparatus 100 can provide a UI for causing the three screens tocooperate on a large screen than the other two screens.

Data such as a video image, a stationary image, text content, or thelike as an entity of the operation target object is exchanged betweenthe information processing apparatus 100 and a corresponding ownedterminal in the background in that the user is performing a touchoperation on the screen or the user is performing an action such ascausing the owned terminal to approach the information processingapparatus 100, for example. Furthermore, a cloud server or the like isinstalled on the external network, and the three screens can benefitfrom cloud computing through the information processing apparatus 100such that the three screens can use a computation ability of the cloudserver, for example.

Hereinafter, some applications of the information processing apparatus100 will be described in order.

B. Simultaneous Operations by a Plurality of Users on Large Screen

The information processing apparatus 100 can be simultaneously operatedby a plurality of users on the large screen. Specifically, the proximitysensor 511 for detecting user presence or a user state is provided ateach of the four side edge portions of the large screen, and useroccupying areas and a common area are set in the screen in accordancewith the user arrangement to make it possible for the plurality of usersto comfortably and efficiently perform simultaneous operations.

If the screen of the information processing apparatus 100 is enlarged,room for spaces on which the plurality of users simultaneously performtouch inputs is generated in the Tabletop state. It is possible torealize comfortable and efficient simultaneous operations by theplurality of users by setting the user occupying area for each user andthe common area in the screen as described above.

An operation right of the operation target object placed in a useroccupying area is given to a corresponding user. When the user displacesthe operation target object from the common area or a user occupyingarea of another user to their own user occupying area, the operationright of the operation target object is moved to the user. In addition,when the operation target object enters their own user occupying area,the display is automatically changed such that the operation targetobject is in the direction in which the operation target object facesthe user themselves.

In a case in which an operation target object is displaced between useroccupying areas, the operation target object is physically displaced ina natural operation in accordance with a touch position at which adisplacement operation is performed. In addition, users can perform anoperation for dividing or copying one operation target object bymutually pulling the operation target object by users.

Main functions of the computation unit 120 when the application isimplemented includes optimization of an operation target object and UIgeneration based on a user detection result by the input interface unit110, a screen touch detection result, and data received from the deviceowned by the user. FIG. 7 shows an internal configuration for thecomputation unit 120 performing processing on an operation targetobject. The computation unit 120 is provided with a monitor areadividing unit 710, an object optimal processing unit 720, and anequipment coordination data exchange processing unit 630.

When user position information is obtained from the input interfaceunifying unit 520, the monitor area dividing unit 710 refers to devicedatabase 711 relating to shapes, sensor arrangement, and the like andarea pattern database 711 stored on the storage unit 140 and sets theaforementioned user occupying areas and the common area on the screen.Then, the monitor area setting unit 710 passes the set area informationto the object optimal processing unit 720 and the equipment coordinationdata exchanging unit 730. A processing procedure for monitor areadivision will be described in detail later.

The object optimal processing unit 720 inputs information on operationswhich the user performs on the operation target object on the screenfrom the input interface unifying unit 520. Then, the object optimalprocessing unit 720 performs optimal processing on the operation targetobject, that is, rotates, displaces, displays, divides, or copies theoperation target object, on which the user has performed an operation,corresponding to the user operation based on an optimal processingalgorithm 721 loaded from the storage unit 140, and outputs theoperation target object, on which the optimal processing has beenperformed, to the screen of the display unit 603. The optimal processingfor the operation target object will be described in detail later.

The equipment coordination data exchanging unit 730 inputs positioninformation of the user and the device owned by the user and dataexchanged with the device from the input interface unifying unit 520.Then, the equipment coordination data exchanging unit 730 performs dataexchange processing in cooperation with the device owned by the userbased on an exchange processing algorithm 731 loaded from the storageunit 140. In addition, the object optimal processing unit 720 performsoptimal processing on the operation target object corresponding to thedata exchange processing. For example, the object optimal processingunit 720 performs optimal processing of the operation target objectaccompanying the data exchange in cooperation with the device owned bythe user, that is, rotates, displaces, displays, or copies the operationtarget object relating to the exchanged data, and outputs the operationtarget object, on which the optimal processing has been performed, tothe screen of the display unit 603. The optimal processing for theoperation target object accompanying the device cooperation will bedescribed in detail later.

Subsequently, detailed description will be given of monitor areadivision processing. Although the monitor area division is assumed to beprocessing mainly in a use state in which the information processingapparatus 100 is commonly used by a plurality of users in the Tabletopstate, it is a matter of course that the processing may be performed ina use state in which the information processing apparatus 100 iscommonly used by a plurality of users in the Wall state.

When user presence is detected through the input interface unifying unit520, the monitor area dividing unit 710 allocates a user occupying areafor the user in the screen. FIG. 8 shows a state in which the monitorarea dividing unit 710 sets a user occupying area A for the user A inthe screen in response to detecting presence of the user A by adetection signal from the proximity sensor 511 (or the distance sensor507) installed at the side edge portion of the screen. When presence ofonly one user has been detected, the entire screen may be set to theuser occupying area for the user as shown in the drawing.

Here, when the user occupying area A is set, the object optimalprocessing unit 720 shifts a direction of each operation target objectin the user occupying area A so as to face the user based on theposition information of the user A obtained through the input interfaceunifying unit 520. FIG. 9A shows a state in which each of the operationtarget objects #1 to #6 is randomly oriented before the user occupyingarea A is set. In the drawing, it should be understood that inclinationof each character “#1”, “#2”, . . . in the object represents a directionof the object in the drawing. In addition, FIG. 9B shows a state inwhich directions of all operation target objects #1 to #6 in this areaare shifted to a direction in which the operation target objects #1 to#6 face a user A by setting the user occupying area A for the user A.

When the presence of only the user A has been detected, the entirescreen may be set to the user occupying area A for the user A. On theother hand, when the presence of two or more users has been detected, itis preferable to set a common area which can be commonly used by theusers in order that the users perform cooperative work (collaborativeaction).

FIG. 10 shows a state in which the monitor area dividing unit 710additionally sets a user occupying area B for the user B and a commonarea in the screen since presence of user B has been detected at anadjacent side edge of the screen by a detection signal of the proximitysensor 511 or the distance sensor 507, in addition to the user A. Theuser occupying area A for the user A retreats to a position where theuser A is present, and the user occupying area B for the user B appearsnear a position where the user B is present, based on positioninformation of the users A and B. In addition, since the presence of theuser B has been newly detected, a wave-like detection indicator is alsodisplayed in the user occupying area B. Then, all area other than theuser occupying area A and the user occupying area B in the screenbecomes the common area. The user occupying area B may be enabled at atiming at which an arbitrary operation target object in the useroccupying area B is touched for the first time after the user occupyingarea B is newly set in the screen when the user B approaches theinformation processing apparatus 100. Although not shown in FIG. 10, adirection of each operation target object in the area which has newlybecome the user occupying area B is shifted so as to face the user B attiming at which the user occupying area B is set or at timing at whichthe user occupying area B is enabled.

FIG. 11 shows a state in which the monitor area dividing unit 710additionally sets a user occupying area D for a user D in the screennear a position where the D is present since presence of the user D hasbeen detected at still another side edge of the screen in addition tothe users A and B. In the user occupying area D, a wave-like detectionindicator is displayed to express that presence of the user D has beennewly detected. In addition, FIG. 12 shows a state in which the monitorarea dividing unit 710 additionally sets a user occupying area C for auser C in the screen near a position where the user C is present sincepresence of the user C has been detected at still another side edge ofthe screen in addition to the users A, B, and D. In the user occupyingarea C, a wave-like detection indicator is displayed to express that thepresence of the user C has been newly detected.

In addition, area division patterns for the user occupying areas and thecommon areas in FIGS. 8 to 12 are shown only for illustrative purposes.The area division pattern depends on a shape of the screen, a number ofusers which have been detected to be present, and arrangement thereof.Area division pattern database 611 accumulates information relating toarea division patterns in accordance with a shape and a size of thescreen and a number of users. In addition, device database 711accumulates information on a shape and a size of the screen used in theinformation processing apparatus 100. Then, when position information ona detected user is input through the input interface unifying unit 520,the monitor area dividing unit 710 reads the shape and the size of thescreen from the device database 711 and refers to an area divisionpattern database 712 about a corresponding area division pattern. FIGS.13A to 13E show examples of the area division patterns in which thescreen is divided into user occupying areas for users in accordance witha shape and a size of the screen and a number of the users.

FIG. 14 shows a processing procedure for the monitor area divisionexecuted by the monitor area dividing unit 710 in the form of aflowchart.

First, the monitor area dividing unit 710 checks whether or not usersare present near the screen based on a signal analysis result of adetection signal by the proximity sensor 511 or the distance sensor 507(Step S1401).

When it has been detected that the users are present (Yes in StepS1401), the monitor area dividing unit 710 subsequently obtains a numberof the present users (Step S1402) and further obtains positions wherethe users are present (Step S1403). The processing in Steps S1401 toS1403 is implemented based on user position information passed from theinput interface unifying unit 520.

Subsequently, the monitor area dividing unit 710 refers to the devicedatabase 711 to obtain device information such as a shape of the screenof the display unit 603 used in the information processing apparatus 100and arrangement of the proximity sensors 511, combines this with theuser position information, and refers to the area division patterndatabase 712 about a corresponding area division pattern (Step S1404).

Then, the monitor area dividing unit 710 sets a user occupying area foreach user and a common area on the screen (Step S1405) based on theobtained area division pattern and completes this processing routine.

Next, detailed description will be given of object optimal processing bythe object optimal processing unit 720.

When information on an operation which a user has performed on theoperation target object on the screen is input through the inputinterface unifying unit 520, the object optimal processing unit 720performs display processing, that is, rotates, displaces, displays,divides, or copies the operation target object on the screen in responseto the user operation. The processing for rotating, displacing,displaying, dividing, or copying the operation target object in responseto a user operation such as dragging or throwing is similar to a GUIoperation on a desktop screen on a computer.

According to this embodiment, the user occupying areas and the commonarea are set on the screen, and the object optimal processing unit 720performs optimal processing on the display in accordance with areaswhere operation target objects are present. A representative example ofthe optimal processing is processing for shifting a direction of eachoperation target object in a user occupying area so as to face a userthereof.

FIG. 15 shows a state in which the object optimal processing unit 720performs processing for automatically rotating an operation targetobject #1 in a direction in which the object faces the user A at amoment at which the object in the common area is dragged or throwntoward the user occupying area A for the user A and a part or a centercoordinate of the object enters to the user occupying area A. Inaddition, FIG. 15 also shows a state in which processing is performedfor automatically rotating an operation target object #2 in a directionin which the object faces the user A at a moment at which the object inthe user occupying area B for the user B is dragged or thrown toward theuser occupying area A for the user A and a part or a center coordinateof the object enters to the user occupying area A.

As shown in FIG. 10, when the user B approaches the informationprocessing apparatus 100, the user occupying area B is newly set on thescreen near the user B. If an operation target object #3 which hasoriginally faced the user A is present in the area which becomes theuser occupying area B (see FIG. 16A), the object optimal processing unit720 immediately performs processing for automatically rotating theoperation target object #3 in a direction in which the operation targetobject #3 faces the user B when the user occupying area B newly appears(see FIG. 16B).

Alternatively, the user occupying area B may be enabled at a timing atwhich an arbitrary operation target object in the user occupying area Bis touched for the first time after the user occupying area B is newlyset when the user B approaches the information processing apparatus 100,instead of immediately performing the processing for rotating theoperation target object. In such a case, processing may be performed forsimultaneously rotating all operation target objects in the useroccupying area B in a direction in which the operation target objectsface the user B at timing at which the user occupying area B is enabled.

The object optimal processing unit 720 can perform the optimalprocessing for the operation target object based on user operationinformation obtained through the input interface unifying unit 520 andarea information passed from the monitor area dividing unit 710. FIG. 17shows an optimal processing procedure for the operation target objectexecuted by the object optimal processing unit 720 in the form of aflowchart.

When the position information of an operation target object which theuser has operated is passed from the input interface unifying unit 520,and the divided monitor area information is obtained from the monitorarea dividing unit 710, the object optimal processing unit 720 checks inwhich area the operation target object operated by the user is present(Step S1701).

Here, when the operation target object operated by the user is presentin the user occupying area, the object optimal processing unit 720checks whether the operation target object is oriented in a direction inwhich the operation target object faces the user in the user occupyingarea (Step S1702).

Then, when the operation target object is not oriented in the directionin which the operation target object faces the user (No in Step S1702),the object optimal processing unit 720 performs rotation processing onthe operation target object so as to be in the direction in which theoperation target object faces the user in the user occupying area (StepS1703).

The rotation direction may be controlled in accordance with a positionat which the user performs a touch operation on the operation targetobject when the user displaces the operation target object from thecommon area or a use occupying area for another user to their own useroccupying area by dragging or throwing. FIG. 18 shows a state in whichan operation target object is rotated about a position of the center ofgravity in a clockwise direction so as to be in the direction in whichthe operation target object faces the user at a timing at which the usertouches the operation target object on the right side of the position ofthe center of gravity and displaces the operation target object bydragging or throwing to cause the operation target object to enter tothe user occupying area. In addition, FIG. 19 shows a state in which theoperation target object is rotated about the position of the center ofgravity in a counterclockwise direction so as to be in the direction inwhich the operation target object faces the user when the user touchesthe operation target object on the left side of the position of thecenter of gravity and displaces the operation target object by draggingor throwing.

As shown in FIGS. 18 and 19, it is possible to provide a naturaloperational feeling to the user by shifting the rotation directions ofthe operation target object with reference to the position of the centerof gravity.

Next, description will be given of equipment coordination data exchangeprocessing by the equipment coordination data exchanging unit 730.

As shown in FIG. 4, the information processing apparatus 100 cancommunicate with other devices such as a mobile terminal owned by a userthrough the communication unit 150. For example, data such as a videoimage, a stationary image, or text content as an entity of an operationtarget object is exchanged between the information processing apparatus100 and the owned terminal in background in which the user is performinga touch operation on the screen or the user performs an action such ascausing the owned terminal to approach the information processingapparatus 100.

FIG. 20 shows an example of an interaction in which an operation targetobject is given and received between the information processingapparatus 100 and the terminal owned by the user. In the example shownin the drawing, a UI expression in which an operation target objectappears from the vicinity of the terminal and flows into the useroccupying area A in response to the user A causing the terminal owned bythemselves to approach the periphery of the user occupying area Aallocated to the user themselves.

The information processing apparatus 100 can detect that the terminalowned by the user has approached the vicinity of the user occupying areaA based on a signal analysis result of a detection signal by thevery-short-range communication unit 513 and a recognition result of auser image captured by the camera unit 503. In addition, the equipmentcoordination data exchanging unit 730 may be designed to specify whetheror not the user has data to be transmitted to the information processingapparatus 100 and what the data to be transmitted is, based on contextbetween the user A and the information processing apparatus 100 (orcommunication between the user A and other users through the informationprocessing apparatus 100). Then, when there is data to be transmitted,the equipment coordination data exchanging unit 730 exchange data suchas a video image, a stationary image, or text content as an entity ofthe operation target object between the information processing apparatus100 and the corresponding owned terminal in background of an action thatthe terminal owned by the user is made to approach the vicinity of theuser occupying area A.

While the equipment coordination data exchanging unit 730 exchanges datawith the terminal owned by the user in the background, the UI expressionthat the operation target object appears from the device owned by theuser is performed on the screen of the display unit 603 by objectoptimal processing by the object optimal processing unit 720. FIG. 20shows an example of UI expression that the operation target object flowsfrom the terminal owned by the user A into the corresponding useroccupying area A.

FIG. 21 shows a processing procedure for equipment coordination dataexchange executed by the device data exchanging unit 730 in the form ofa flowchart. The processing by the equipment coordination dataexchanging unit 730 is activated when a terminal owned by a certain useris made to approach the vicinity of their own user occupying area, basedon a signal analysis result of a detection signal by thevery-short-range communication unit 513.

The equipment coordination data exchanging unit 730 checks whether ornot a communicating terminal owned by a user is present based on asignal analysis result of a detection signal by the very-short-rangecommunication unit 513 (Step S2101).

When a communicating terminal owned by a user is present (Yes in StepS2101), the equipment coordination data exchanging unit 730 obtains aposition where the terminal is present based on the signal analysisresult of the detection signal by the very-short-range communicationunit 513 (Step S2102).

Then, the equipment coordination data exchanging unit 730 checks whetheror not there is data exchanged with the terminal owned by the user (StepS2103).

When there is data exchanged with the terminal owned by the user (Yes inStep S2103), the equipment coordination data exchanging unit 730performs UI display (see FIG. 20) of the operation target object inaccordance with the terminal position based on the exchange processingalgorithm 731. In addition, the equipment coordination data exchangingunit 730 exchanges data as entity of the operation target object withthe terminal in background of the UI display (Step S2104).

As shown in FIGS. 20 and 21, the operation target object obtained by theinformation processing apparatus 100 from the terminal owned by the useris arranged in a user occupying area for a corresponding user.Furthermore, when data communication is performed between users, anoperation for displacing the operation target object may be performedbetween mutual user occupying areas. FIG. 22 shows a state in which anoperation target object maintained by the user B in the user occupyingarea B is copied by the user A to their own user occupying area A.Alternatively, the operation target object may not be copied and may bedivided.

When the operation target object copied on the screen is content such asa video image or a stationary image, other independent data is simplycreated. In addition, when the copied operation target object is awindow of an application, another window of an application which can becommonly operated by the user who originally holds the operation targetobject and a user of a destination of the copying is created.

C. Optimal Selection of Input Means and Display GUI in Accordance withUser Position

The information processing apparatus 100 is provided with the distancesensor 507 and the proximity sensor 511 and can detect a distance fromthe main body of the information processing apparatus 100, namely thescreen to a user while hung on the wall and used as shown in FIGS. 1 and3, for example.

In addition, the information processing apparatus 100 is provided withthe touch detection unit 509, the proximity sensor 511, the camera unit503, and the remote control receiving unit 501 and can provide aplurality of input means such as touching the screen, proximity, gestureusing a hand or the like, a remote controller, and an indirect operationdepending on a user state to the user. The operation by each input meanscan be applied or cannot be applied in accordance with the distance fromthe main body of the information processing apparatus 100, namely thescreen to the user. For example, a user within a range of cm or closerfrom the main body of the information processing apparatus 100 canreliably operate an operation target object by directly touching thescreen. In a case of a user within a range of 2 m or closer from themain body of the information processing apparatus 100, it is possible toprecisely capture motions of a face and a hand by performing recognitionprocessing on an image captured by the camera unit 503 while the user islocated too far to directly touch the screen, and therefore, a gestureinput can be made. In a case of a user who is distant away from the mainbody of the information processing apparatus 100 by 2 m or more, aremote control signal reliably reaches while precision of imagerecognition is lowered, and therefore, a remote control operation can bemade. Furthermore, GUI display with optimal frameworks of operationtarget objects to be displayed on the screen, optimal informationdensity, and the like is also changed depending on the distance to theuser.

According to the embodiment, the information processing apparatus 100 isdesigned such that one of the plurality of input means is automaticallyselected depending on the user position or the distance to the user andthe GUI display is automatically selected or adjusted in accordance withthe user position, which enhances user convenience.

FIG. 23 shows an internal configuration for the computation unit 120performing optimization processing in accordance with a distance to auser. The computation unit 120 is provided with a display GUIoptimization unit 2310, an input means optimization unit 2320, and adistance detecting scheme shifting unit 2330.

The display GUI optimization unit 2310 performs optimization processingfor GUI display with optimal frameworks of operation target objects tobe displayed on the screen of the display unit 603 and optimalinformation density in accordance with a user position and a user state.

Here, the user position is obtained based on a distance detecting schemeshifted by the distance detecting scheme shifting unit 2330. When theuser position approach, personal authentication through face recognitionof an image captured by the camera unit 503, proximity communicationwith the terminal owned by the user, or the like can be performed. Inaddition, the user state is specified based on image recognition of animage captured by the camera unit 503 or signal analysis by the distancesensor 507. The user states are roughly classified into two states,namely a state in which “there is a user (present)” and a state in which“there is no user (not present)”. The state in which “there is a user”is broadly classified into two states, namely a state in which “the useris watching the television (the screen of the display unit 603)(watching)” and a state in which “the user is not watching thetelevision (not watching)”. Furthermore, the state in which “the user iswatching the television” is broken down into two states, namely a statein which “the user is operating the television (operating)” and a statein which “the user is not operating the television (not operating)”.

In determining the user state, the display GUI optimization unit 2310refers to the device input means database in the storage unit 140. Inaddition, in optimizing the display GUI in accordance with thedetermined user position and the user state, GUI display (frameworks,density) database and content database in the storage unit 140 arereferred to.

FIG. 24A is a diagram showing, in a table, GUI display optimizationprocessing by a display GUI optimization unit 2310 in accordance with auser position and a user state. In addition, FIGS. 24B to 24E showscreen shift of the information processing apparatus 100 in accordancewith the user position and the user state.

In the state in which “there is no user”, the display GUI optimizationunit 2310 stops screen display by the display unit 603 and waits untilthe presence of a user is detected (see FIG. 24B).

In the state in which “there is a user” but “the user is not watchingthe television”, the display GUI optimization unit 2310 selects“automatic zapping” as an optimal display GUI (see FIG. 24C). Theautomatic zapping is for randomly displaying various operation targetobjects to attract the interest of the user to encourage the user toview the television. The operation target object used in zappingincludes a plurality of operation target objects such as network contentobtained from the communication unit 150 through the Internet ande-mails and messages from other users as well as television broadcastingprogram content received by the television tuner unit 170 which thedisplay GUI optimization unit 2310 selects based on the contentdatabase.

FIG. 25A shows an example of a display GUI being automatically zapped.In addition, the display GUI optimization unit 2310 may momentarilychange a position and a size (that is, a degree of exposure) of eachoperation target object displayed on the screen as shown in FIG. 25B towork on subconscious awareness of the user. In addition, when the userposition approaches and personal authentication can be performed, thedisplay GUI optimization unit 2310 may use information on the recognizedperson to sort out the operation target objects to be automaticallyzapped.

Even in the state in which “the user is watching the television” but“the user is not operating the television”, the display GUI optimizationunit 2310 selects the “automatic zapping” as an optimal display GUI (seeFIG. 24D). Unlike the above, however, a plurality of operation targetobjects selected based on the content database are regularly arranged ina column format or the like as shown in FIG. 26 to make it easy to checkthe display contents of individual operation target objects. Inaddition, when the user position approaches and personal authenticationcan be performed, the display GUI optimization unit 2310 may useinformation on the recognized person to sort out operation targetobjects to be automatically zapped. In addition, the GUI optimizationunit 2310 may control information density of the display GUI inaccordance with the user position such that the information density ofthe GUI is suppressed when the user is present at a distant place andthe information density of the GUI is enhanced when the user approaches.

On the other hand, in the state in which “the user is watching thetelevision” and “the user is operating the television”, the user isoperating the information processing apparatus 100 with the use of theinput means optimized by the input means optimization unit 2320 (seeFIG. 24E). The optimal input means in this state is transmission of aremote control signal to the remote control receiving unit 501, gestureto the camera unit 503, contact to the touch panel detected by the touchdetection unit 509, audio input to the microphone 505, proximity inputto the proximity sensor 511, and the like, for example. The display GUIoptimization unit 2310 displays operation target objects in the columnformat as an optimal display GUI in accordance with a user inputoperation and makes it possible to perform scrolling or a selectionoperation of an operation target object in response to a user operation.A cursor is displayed at a position, which is designated by the inputmeans, on the screen as shown in FIG. 27A. Since operation targetobjects on which the cursor is not placed can be considered that theuser is not paying attention thereto, an illumination level may belowered as shown by hatching in the drawing to express contract withrespect to the focused operation target object (in the drawing, thecursor is placed on an operation target object #3 which the user toucheswith a tip of a finger). In addition, entire screen display (or enlargeddisplay to an available maximum size) of the operation target object maybe performed when the user selects the operation target object on whichthe cursor is placed as shown in FIG. 27B (in the drawing, the selectedoperation target object #3 is enlarged and displayed).

The input means optimization unit 2320 performs optimization of theinput means with which the user performs an operation on the informationprocessing apparatus 100 in accordance with the user position and theuser state.

As described above, the user position can be obtained based on thedistance detecting scheme shifted by the distance detecting schemeshifting unit 2330. When the user position approaches, personalauthentication through face recognition of an image captured by thecamera unit 503, proximity communication with the terminal owned by theuser, or the like can be performed. In addition, the user state isspecified by image recognition of an image captured by the camera unit503 or signal analysis by the distance sensor 507.

The input means optimization unit 2320 refers to the device input meansdatabase in the storage unit 140 in determining the user state.

FIG. 28 shows, in a table, input means optimization processing by theinput means optimization unit 2320 in accordance with the user positionand the user state.

In the state in which “there is no user”, the state in which “there is auser” but “the user is not watching the television”, and in the state inwhich “the user is watching the television” but “the user is notoperating the television”, the input means optimization unit 2320 waitsuntil a user operation is started.

Then, in the state in which “the user is watching the television” andalso “the user is operating the television”, the input meansoptimization unit 2320 optimizes each input means mainly in accordancewith the user position. The input means includes a remote control inputto the remote control receiving unit 501, a gesture input to the cameraunit 503, a touch input detected by the touch detection unit 509, anaudio input to the microphone 505, a proximity input to the proximitysensor 511, and the like.

The remote control receiving unit 501 activates all user positions (thatis, nearly all the time) and waits for receiving the remote controlsignal.

Recognition precision of an image captured by the camera unit 503 isgradually lowered as the user is positioned at a further place. Inaddition, when the user excessively approaches, the figure of the useris easily deviated from a field of view of the camera unit 503. Thus,the input means optimization unit 2320 turns on the gesture input to thecamera unit 503 within a range of the user position from several tenscentimeters to several meters.

Touching the touch panel superimposed on the screen of the display unit603 is limited to a range which a hand of the user can reach. Thus, theinput means optimization unit 2320 turns on the touch input to the touchdetection unit 509 within a range of the user position up to severaltens of centimeters. In addition, the proximity sensor 511 can detectthe user at several tens of centimeters even without touching.Accordingly, the input means optimization unit 2320 turns on theproximity input up to a user position which is further than the positionfor the touch input.

Recognition precision on the input audio to the microphone unit 505 isgradually lowered as the user is positioned at a further place. Thus,the input means optimization unit 2320 turns on the gesture input to thecamera unit 503 within a range of the user position up to severalmeters.

The distance detecting scheme shifting unit 2330 performs processing forshifting a scheme, based on which the information processing apparatus100 detects the distance to the user and the user position, inaccordance with the user position.

The distance detecting scheme shifting unit 2330 refers to coveragedatabase for each detection scheme in the storage unit 140 indetermining the user state.

FIG. 29 shows, in a table, distance detection scheme shifting processingby the distance detecting scheme shifting unit 2330 in accordance withthe user position.

The distance sensor 507 is configured by a simple low power consumptionsensor element such as a PSD sensor, a pyroelectric sensor, or a simplecamera. The distance detecting scheme shifting unit 2330 constantlyturns on the distance sensor 507 to constantly monitor whether a user ispresent in a radium from 5 to 10 meters, for example, from theinformation processing apparatus 100.

When a monocular type is employed for the camera unit 503, the imagerecognition unit 504 performs user motion recognition, face recognition,human body recognition by background differencing. The distancedetecting scheme shifting unit 2330 turns on a recognition (distancedetection) function by the image recognition unit 504 within a range ofthe user position from 70 centimeters to 6 meters in which sufficientrecognition prevision can be obtained based on a captured image.

In addition, when a two-lens type or an active type is employed for thecamera unit 503, the image recognition unit 504 can have sufficientrecognition precision from 60 centimeters to 5 meters, which is slightlycloser, and the distance detecting scheme shifting unit 2330 turns onthe recognition (distance detection) function by the image recognitionunit 504 within the range of the user position.

In addition, when the user excessively approaches, the figure of theuser is easily deviated from the field of view of the camera unit 503.Thus, the distance detecting scheme shifting unit 2330 may turn off thecamera unit 503 and the image recognition unit 504 when the userexcessively approaches.

Touching the touch panel superimposed on the screen of the display unit603 is limited to a range which a hand of the user can reach. Thus, thedistance detecting scheme shifting unit 2330 turns on the distancedetecting function by the touch detection unit 509 within a range of theuser position up to several tens of centimeters. In addition, theproximity sensor 511 can detect a user up to several tens of centimeterseven without touching. Accordingly, the distance detecting schemeshifting unit 2330 turns on the distance detecting function by theproximity sensor 511 up to a user position which is further than theposition for the touch input.

As a design argument of the information processing apparatus 100provided with a plurality of distance detecting schemes, it ispreferable to use a low power consumption device since a distancedetecting scheme for detecting a distant place at several meters or overten meters is for the purpose of checking for the presence of a user andshould be constantly turned on. On the other hand, since a distancedetecting scheme for detecting in a close distance within one metersignificantly consumes power for recognition processing although it ispossible to obtain high density information and have a recognitionfunction such as face recognition and human body recognition, it ispreferable to turn off the function at a distance at which sufficientrecognition prevision cannot be obtained.

D. Real Size Display of Object in Accordance with Monitor Performance

According to a conventional subject display system, an image of anactual object is displayed on the screen without considering real sizeinformation. For this reason, the size of the object displayed varies inaccordance with a size or resolution (dpi) of the screen. For example, ahorizontal width a′ when a bag whose horizontal width is a centimetersis displayed on a 32-inch monitor and a horizontal width a″ when the bagis displayed on a 50-inch monitor are different from each other(a≠a′≠a″) (see FIG. 30).

In addition, images of a plurality of objects are simultaneouslydisplayed on a same monitor screen, then magnitude relationship of theobjects are not correctly displayed without considering the real sizeinformation of the objects. For example, when a bag has a horizontalwidth of a centimeters and a pouch has a horizontal width of bcentimeters are simultaneously displayed on a same monitor screen, thebag is displayed at a size of a′ centimeters while the pouch isdisplayed at a size of b′ centimeters, and the mutual magnituderelationship is not correctly displayed (a:b≠a′:b′) (see FIG. 31).

For example, if a sample image of a product is not reconstructed in areal size when the product is purchased online, a user cannot accuratelyfit the product to the own figure, and there is a concern in that theuser may erroneously purchase the product. In addition, if magnituderelationships of sample images are not correctly displayed when a usertries to simultaneously purchase a plurality of products on line and thesample images of the products are simultaneously displayed on thescreen, the user cannot accurately combine and fit the products, andthere is a concern in that the user may purchase the products in animproper combination.

On the other hand, the information processing apparatus 100 is designedto manage real size information of an object to be displayed andinformation in a size and resolution (pixel pitch) of the screen of thedisplay unit 603 and always display an image of the object at a realsize on the screen even if the sizes of the objects and the screen arechanged.

FIG. 32 shows an internal configuration for the computation unit 120performing object real size display processing in accordance with amonitor performance. The computation unit 120 is provided with a realsize display unit 3210, a real size estimation unit 3220, and a realsize expansion unit 3230. However, there may also be a case in which atleast one functional block among the real size display unit 3210, thereal size estimation unit 3220, and the real size expansion unit 3230 isimplemented on the cloud server connected via the communication unit150.

In simultaneously displaying images of a plurality of objects on a samemonitor screen, the real size display unit 3210 constantly displays theimages at real sizes in accordance with a size and resolution (pixelpitch) of the screen of the display unit 603 in consideration of thereal size information of each object. In simultaneously displaying theimages of the plurality of objects on the screen of the display unit603, the real size display unit 3210 correctly displays the magnituderelationships of the objects.

The real size display unit 3210 reads monitor specifications such as asize and a resolution (pixel pitch) of the screen of the display unit603 from the storage unit 140. In addition, the real size display unit3210 obtains monitor states such as a direction and inclination of thescreen of the display unit 603 from the rotation and attachmentmechanism unit 180.

In addition, the real size display unit 3210 reads an image of an objectto be displayed from an object image database in the storage unit 140and reads real size information of the object from the object real sizedatabase. However, there may also be a case in which the object imagedatabase and the object real size database are on a database serverconnected through the communication unit 150.

Then, the real size display unit 3210 performs conversion processing onthe image of the object based on the monitor performance and the monitorstate such that the object to be displayed is displayed at a real sizeon the screen of the display unit 603 (or mutual magnitude relationshipsof a plurality of objects are correct). That is, a=a′=a″ is satisfied asshown in FIG. 33 even when an image of a same object is displayed onscreens with different monitor specifications.

In addition, when the real size display unit 3210 simultaneouslydisplays two objects with different real sizes on a same screen,a:b=a′:b′, that is, the mutual magnitude relationship is correctlydisplayed as shown in FIG. 34.

For example, when a user purchases a product on line through display ofa sample image, the information processing apparatus 100 can implementreal size display of an object and display a plurality of sample imagesin correct magnitude relationships as described above, and therefore,the user can accurately fit the product, and there is less concern inthat the user may erroneously select the product.

An example in which real size display of an object image by the realsize display unit 3210 is applied to an application for online shoppingwill be additionally described. In response to a user touching an imageof a desired product on a display screen of a brochure, the image of theproduct is shifted to real size display (see FIG. 35). In addition, itis possible to change a direction of the object in the real size anddisplay the object by rotating or performing posture conversion on theimage displayed in the real size in response to the touch operation bythe user (see FIG. 36).

In addition, the real size estimation unit 3220 performs processing forestimating a real size of an object, for which real size informationcannot be obtained even by referring to the object real size database,such as a person imaged by the camera unit 503. For example, when anobject for which a real size is estimated is a user's face, a real sizeof the user is estimated based on user face data such as the size of theuser's face, age, and a direction which have been obtained by the imagerecognition unit 504 performing image recognition on the image capturedby the camera unit 503 and a user position obtained by the distancedetecting scheme shifted by the distance detecting scheme shifting unit2330.

The estimated user real size information is feedback to the real sizedisplay unit 3210 and stored on the object image database, for example.Then, the real size information estimated from the user face data isused for real size display in accordance with the later monitorperformance by the real size display unit 3210.

For example, when an operation target object including a captured imageof a target (baby) is displayed as shown in FIG. 37A, the real sizeestimation unit 3220 estimates the real size based on face data.Thereafter, the target is not enlarged to a size which exceeds the realsize of the target as shown in FIG. 37B even if the user tries toenlarge and display the operation target object by a touch operation orthe like. That is, the image of the baby is not enlarged to an unnaturalsize, and reality of the movie is maintained.

In addition, when network content and content captured by the cameraunit 503 are displayed on the screen of the display unit 603 in aparallel manner or in a superimposed manner, it is possible to implementbalanced parallel display or superimposed display by performingnormalization processing on the movie of the content based on theestimated real sizes.

Furthermore, the real size expansion unit 3230 implements real sizedisplay of an object, which is implemented on the screen of the displayunit 603 by the real size display unit 3210, even in a 3D direction,that is a depth direction. In a case of performing 3D display based on abinocular scheme or a light beam reconstruction scheme, a desired effectcan be achieved only at a viewing position assumed in generating the 3Dmovie. Based on an omnidirectional light beam reconstruction scheme,actual size display can be performed at an arbitrary position.

In addition, the real size expansion unit 3230 can achieve the same realsize display at an arbitrary position even in the binocular scheme andthe light beam reconstruction scheme only in the horizontal direction bydetecting a position of an observing point of the user and correctingthe 3D movie with respect to the position.

See Japanese Unexamined Patent Application Publication No. 2002-300602,Japanese Unexamined Patent Application Publication No. 2005-149127, andJapanese Unexamined Patent Application Publication No. 2005-142957, allof which have already been assigned to the applicant, for example.

E. Simultaneous Display of Image Group

There is a case in which movie content of a plurality of sources issimultaneously displayed on a same screen in a parallel manner or in asuperimposed manner in a display system. For example, it is possible toexemplify (1) a case in which a plurality of users perform a video chat,(2) a case in which a movie of an instructor reproduced from a recordingmedium such as a DVD (or streaming-reproduced through a network) and amovie of a user themselves captured by the camera unit 503 aresimultaneously displayed in a lesson of yoga or the like, and (3) a casein which a sample image of a product and a movie of a user themselvescaptured by the camera unit 503 are superimposed and displayed forfitting in online shopping.

In any of the above cases (1) to (3), the user cannot appropriately usethe displayed movie if a magnitude relationship of the simultaneouslydisplayed images is not correctly displayed. For example, if sizes andthe positions of the users who perform a video chat are not adjusted(FIG. 38A), face-to-face contact with the counterpart of the chat isdegraded, and the users cannot enjoy the conversation. In addition, whensizes and positions of the figures of the user and the instructor arenot adjusted (FIG. 39A), it is difficult for the user to find adifference between their own motion and motion of the instructor, andthe user cannot know a point to be modified or improved and cannotattain sufficient progress in the lesson. In addition, if the sampleimage of the product is not superimposed on an appropriate position in acorrect magnitude relationship with the movie of the user themselves whohold a pose of picking up the product with a hand, it is difficult forthe user to determine whether the product suits the user themselves, andthe user cannot accurately perform fitting (FIG. 40A).

On the other hand, the information processing apparatus 100 according tothe embodiment is designed to normalize images with the use ofinformation on a scale of an image, a corresponding area, and the likeand display the images in a parallel manner or in a superimposed mannerwhen movie content of a plurality of sources is arranged in parallel orsuperimposed. In the normalization, image retouch including digitalzooming processing on digital image data such as a stationary image or avideo image is performed. In addition, when one of the images arrangedin parallel or superimposed is an image captured by the camera unit 503,optical control for panning, tilting, zooming, and the like areperformed on the actual camera.

It is possible to simply implement the image normalization processingwith the use of information on a size of a face obtained by facerecognition, age, a direction, and the like and information on a bodyshape and a size obtained by person recognition. In addition, mirroringand rotation processing are automatically performed on one of the imagesto easily find a correspondence relationship with the other image indisplaying a plurality of images in a parallel manner or in asuperimposed manner.

FIG. 38B shows a state in which sizes and positions of faces of theusers performing video chatting are adjusted by normalization processingamong the plurality of images. In addition, FIG. 39B shows a state inwhich the sizes and the positions of the figures of the user and theinstructor displayed in parallel on the screen are adjusted bynormalization processing among a plurality of images. In addition, FIG.40B shows a state in which the sample image of the product issuperimposed and displayed at an appropriate position in a correctmagnitude relationship with the movie of the user themselves who holds apose of picking up the product by normalization processing among aplurality of images. In addition, not only the magnitude relationshipnormalization processing but also mirroring is performed in FIGS. 39Band 40B in order that the user can easily correct their own posture fromthe image captured by the camera unit 503. In addition, there is also acase in which rotation processing is performed as necessary. Inaddition, if the normalization processing can be performed on thefigures of the user and the instructor, the figures can be alsodisplayed in the superimposed manner as shown in FIG. 39C instead ofbeing displayed in the parallel manner as shown in FIG. 39B, and theuser can further easily and visually recognize a difference betweentheir own posture and the posture of the instructor.

FIG. 41 shows an internal configuration for the computation unit 120performing image normalization processing. The computation unit 120 isprovided with an inter-image normalization processing unit 4110, a facenormalization processing unit 4120, and a real size expansion unit 4130.However, there may also be a case in which at least one functional blockamong the inter-image normalization processing unit 4110, the facenormalization processing unit 4120, and the real size expansion unit4130 is implemented on the cloud server connected through thecommunication unit 150.

The inter-image normalization processing unit 4110 performsnormalization processing such that a magnitude relationship between aface image of a user and other objects is correctly displayed among aplurality of images.

The inter-image normalization processing unit 4110 inputs a user imagecaptured by the camera unit 503 through the input interface unifyingunit 520. On this occasion, camera information for panning, tilting,zooming, and the like of the camera unit 503 in imaging the user is alsoobtained. In addition, the inter-image normalization processing unit4110 obtains an image of another object to be displayed with the userimage in a parallel manner or in a superimposed manner and obtains apattern in which the user image and the image of another object arearranged in parallel or superimposed from image database. The imagedatabase is present in the storage unit 140 in some cases and is presenton the database server which is accessed through the communication unit150 in other cases.

Then, the inter-image normalization processing unit 4110 performs imageretouching such as enlargement, rotation, and mirroring on the userimage based on a normalization algorithm such that a magnituderelationship with other objects and a posture become correct, andgenerates camera control information for controlling panning, tilting,zooming, and the like of the camera unit 503 so as to appropriatelycapture the user image. By the processing of the inter-imagenormalization processing unit 4110, the user image is displayed so as tomaintain a correct magnitude relationship with the images of otherobjects as shown in FIG. 40B, for example.

The face normalization processing unit 4120 performs normalizationprocessing such that a user face image captured by the camera unit 503is correctly displayed so as to maintain a magnitude relationship with aface image in other operation target objects (for example, a face of aninstructor in an image reproduced from a recording medium or a face of acounterpart user in a video chat).

The face normalization processing unit 4120 inputs a user image capturedby the camera unit 503 through the input interface unifying unit 520. Onthis occasion, camera information on panning, tilting, zooming, and thelike of the camera unit 503 in imaging the user is also obtained. Inaddition, the face normalization processing unit 4120 obtains faceimages in other operation target objects, which is to be displayed withthe captured user image in a parallel manner or in a superimposedmanner, through the storage unit 140 or the communication unit 150.

Then, the face normalization processing unit 4120 performs imageretouching such as enlargement, rotation, and mirroring on the userimage such that a magnitude relationship of mutual face images becomecorrect, and generates camera control information for controllingpanning, tilting, zooming, and the like of the camera unit 503 so as toappropriately capture the user image. By the processing of the facenormalization processing unit 4120, the user image is displayed so as tomaintain a correct magnitude relationship with other face images asshown in FIGS. 38B, 39B, and 39C, for example.

Furthermore, the real size expansion unit 4130 implements parallel orsuperimposed display of a plurality of images, which is implemented onthe screen of the display unit 603 by the inter-image normalizationprocessing unit 4110 or the inter-image normalization processing unit4110, even in a 3D direction, namely a depth direction. In addition, ina case of performing 3D display based on a binocular scheme or a lightbeam reconstruction scheme only in the horizontal direction, a desiredeffect can be achieved only at a viewing position assumed in generatinga 3D movie. Based on an omnidirectional light beam reconstructionscheme, actual size display can be performed at an arbitrary position.

In addition, the real size expansion unit 4130 can achieve the same realsize display at an arbitrary position even in the binocular scheme andthe light beam reconstruction scheme only in the horizontal direction bydetecting a position of an observing point of the user and correctingthe 3D movie with respect to the position.

See Japanese Unexamined Patent Application Publication No. 2002-300602,Japanese Unexamined Patent Application Publication No. 2005-149127, andJapanese Unexamined Patent Application Publication No. 2005-142957, allof which have already been assigned to the applicant, for example.

F. Movie Content Display Method in Rotated Screen

As described above, the main body of the information processingapparatus 100 according to the embodiment is attached to a wall face ina rotatable and detachable manner by the rotation and attachmentmechanism unit 180, for example. If a rotation operation is performed onthe main body while the power of the information processing apparatus100 is on, that is, while the display unit 603 displays operation targetobjects, rotation processing is performed on the operation targetobjects along with the rotation operation such that a user can observethe operation target object in correct postures.

Hereinafter, description will be given of a method for optimallyadjusting a movie content display mode at an arbitrary rotation angle ofthe main body of the information processing apparatus 100 in a shiftingprocess thereof.

As movie content display modes at arbitrary rotation angles of thescreen in the shifting process thereof, it is possible to exemplifythree modes, namely (1) a display mode for allowing the movie content tobe completely on the screen at an arbitrary rotation angle, (2) adisplay mode for maximizing focused content in the movie content at eachrotation angle, and (3) a display mode for rotating the movie content soas not to generate an invalid area.

FIG. 42 is a diagram showing the display mode in which an entire area ofmovie content is displayed such that the movie content is completely onthe screen at an arbitrary rotation angle while the informationprocessing apparatus 100 (screen) is rotated in the counterclockwisedirection by 90°. In rotating the information processing apparatus 100in the counterclockwise direction by 90° to longitudinally install theinformation processing apparatus 100 while laterally long movie contentis displayed on the laterally installed screen as shown in the drawing,the movie content is downsized, and an invalid area shown in black onthe screen appears. In addition, in the shifting process of the screenfrom the lateral installation to the longitudinal installation, themovie content is minimized.

If at least a part of the movie content is not on the screen, there is aproblem in that the movie content as a copyrighted work loses identity.In the display mode as shown in FIG. 42, the identity as a copyrightedwork is always secured at an arbitrary rotation angle in the shiftingprocess thereof. That is, it can be said that the display mode suitsprotected content.

In addition, FIG. 43 is a diagram showing the display mode in which afocused area in the movie content is maximized at each rotation anglewhile the information processing apparatus 100 (screen) is rotated inthe counterclockwise direction by 90°. In the drawing, an area includingthe target surrounded by a dotted line in the movie content is set as afocused area, and the focused area is maximized at each rotation angle.Since the focused area is vertically long, the movie content is enlargedwhen the lateral installation is changed into the longitudinalinstallation. In addition, the focused area is maximized in a directionof a diagonal of the screen in the shifting process from the lateralinstallation to the longitudinal installation. In addition, an invalidarea shown in black appears on the screen in the shifting process fromthe lateral installation to the longitudinal installation.

As a display mode focusing on a focused area in the movie content, amodified example can also be considered in which the movie content isrotated while the size of the focused area is constantly maintained.Although the focused area appears to be smoothly rotated during therotation of the screen, the invalid area is enlarged.

In addition, FIG. 44 is a diagram showing the display mode in which themovie content is rotated such that an invalid area does not appear whilethe information processing apparatus 100 (screen) is rotated in thecounterclockwise direction by 90°.

FIG. 45 shows a relationship of a zooming rate of the movie content withrespect to a rotation position in the display modes shown in FIGS. 42 to44. In the display mode for allowing the movie content to be completelyin the screen at an arbitrary rotation angle shown in FIG. 42, a largeinvalid area is generated in the shifting process although the contentcan be protected. In addition, there is a concern in that feeling ofstrangeness is given to the user since the movie is downsized in theshifting process. In the display mode for maximizing a focused area inthe movie content at each rotation angle shown in FIG. 43, an invalidarea is generated in the shifting process although it is possible tomore smoothly display the focused area in the shifting process duringwhich the screen is rotated. In addition, in the display mode shown inFIG. 44, the movie content is significantly enlarged in the shiftingprocess although an invalid area is not generated in the shiftingprocess, and there is a concern in that unnatural impression is given toan observing user.

FIG. 46 shows a processing procedure for controlling a display mode ofmovie content by the computation unit 120 when the informationprocessing apparatus 100 (the screen of the display unit 603) in theform of a flowchart. The processing procedure is activated in responseto detection by the rotation and attachment mechanism unit 180, forexample, that the main body of the information processing apparatus 100is being rotated or detected by the triaxial sensor 515 a variation inthe rotation position of the main body of the information processingapparatus 100.

In rotating the information processing apparatus 100 (the screen of thedisplay unit 603), the computation unit 120 first obtains attributeinformation of the movie content being displayed on the screen (StepS4601). Then, it is checked whether the movie content being displayed onthe screen is content protected by a copyright or the like (Step S4602).

Here, when the movie content being displayed on the screen is contentprotected by a copyright or the like (Yes in Step S4602), thecomputation unit 120 selects the display mode for displaying the entirearea of the movie content such that the movie content is completely inthe screen at an arbitrary rotation angle as shown in FIG. 42 (StepS4603).

In addition, when the movie content being displayed on the screen is notcontent protected by a copyright or the like (No in Step S4602), it issubsequently checked whether or not the display mode is a display modedesignated by the user (Step S4604).

When the user has selected the display mode for displaying the entirearea of the movie content, the processing proceeds to Step S4603. Inaddition, when the user has selected the display mode for maximizing anddisplaying a focused area, the processing proceeds to Step S4605. Inaddition, when the user has selected the display mode for not displayingan invalid area, the processing proceeds to Step S4606. Moreover, whenthe user has not selected any of the display modes, a display mode setas a default value is selected among the above three display modes.

FIG. 47 shows an internal configuration for the computation unit 120performing processing for adjusting a display mode of movie content atan arbitrary rotation angle of the main body of the informationprocessing apparatus 100 in the shifting process. The computation unit120 is provided with a display mode selecting unit 4710, a rotationposition input unit 4720, and an image retouching unit 4730 and adjustsreceived television broadcasts or movie content reproduced from amedium.

The display mode selecting unit 4710 selects a display mode for rotatingthe movie content at an arbitrary rotation angle of the main body of theinformation processing apparatus 100 in the shifting process based onthe processing procedure shown in FIG. 46.

The rotation position input unit 4720 inputs a rotation position of themain body of the information processing apparatus 100 (or the screen ofthe display unit 602), which is obtained by the rotation and attachmentmechanism unit 180 or the triaxial sensor 515, through the inputinterface unifying unit 520.

The image retouching unit 4730 performs image retouching based on thedisplay mode selected by the display mode selecting unit 4710 such thatthe received television broadcasts or the movie content reproduced fromthe medium suits the screen of the display unit 603 which is inclined ata rotation angle input to the rotation position input unit 4720.

G. Technique Disclosed in this Specification

The technique disclosed in this specification can be configured asfollows.

(1)An information processing apparatus including:a control unit to control processing on an operation object, wherein anoperation right of the operation object is given to a user, displayed ona display unit based on detection of a user from at least oneinformation signal input to the apparatus from outside of the apparatus.(2)The apparatus according to (1), wherein the control unit sets a useroccupying area in a screen of the display unit for each user detectedfrom the at least one information signal.(3)The apparatus according to (1), wherein the control unit sets a commonarea in a screen of the display unit usable by each user detected fromthe at least one information signal.(4)The apparatus according to (1), wherein the control unit sets (i) a useroccupying area in a screen of the display unit for each user detectedfrom the at least one information signal, and (ii) all area of thescreen other than the user occupying area of the each user detected as acommon area usable by the each user detected.(5)The apparatus according to (1) further comprising:at least one proximity sensor disposed at each of four side edgeportions of a screen of the display unit and at which the at least oneinformation signal is input.(6)The apparatus according to (1), wherein the control unit automaticallyrotates the operation object, when present in a user occupying area setfor the user in a screen of the display unit, to face the user.(7)The apparatus according to (6), wherein the operation object isautomatically rotated in accordance with a predetermined timing.(8)The apparatus according to (1), wherein the control unit rotates theoperation object in accordance with a position, at which the user hasperformed an operation with respect to a position of a center of gravityof the operation object, when the user displaces the operation objectbetween set first and second areas in a screen of the display unit.(9)The apparatus according to (8), wherein the user displaces the operationobject by dragging or throwing.(10)The apparatus according to (1) further comprising: a data exchange unitto exchange data with a terminal owned by the user.(11)The apparatus according to (10), wherein the control unit displays theoperation object on a screen of the display unit, based on the dataexchanged with the terminal.(12)The apparatus according to (1), wherein the control unit controls atleast one of copying or dividing of the operation object, in response toan operation for displacing the operation object.(13)The apparatus according to (1) further comprising:an input interface unit to which the at least one information signalfrom the outside is input.(14)The apparatus according to (1), wherein the control unit controls theprocessing on the operation object based on at least one of a screentouch detection result or data received from a terminal owned by theuser.(15)The apparatus according to (1), wherein the control unit controls theprocessing on the operation object by selecting one of a plurality ofinput units of the apparatus.(16)The apparatus according to (15), wherein the selecting is in accordancewith position of or distance to the user.(17)The apparatus according to (1), wherein the apparatus includes thedisplay unit.(18)The apparatus according to (1), wherein the processing on the operationobject is based on a state of the user.(19)An information processing method including:controlling, by a processor, of processing on an operation object,wherein an operation right of the operation object is given to a user,displayed on a display unit based on detection of a user from at leastone information signal input to an information processing apparatus fromoutside of the apparatus.(20)A non-transitory recording medium recorded with a program executable bya computer, the program including:controlling processing on an operation object, wherein an operationright of the operation object is given to a user, displayed on a displayunit based on detection of a user from at least one information signalinput to an information processing apparatus from outside of theapparatus.(101)An information processing apparatus including: a display unit; a userdetection unit which detects a user who is present in the periphery ofthe display unit; and a computation unit which performs processing on anoperation target object displayed on the display unit in response todetection of the user by the user detection unit.(102)The information processing apparatus according to (101), wherein theuser detection unit is provided with a proximity sensor which isdisposed at each of four side edge portions of a screen of the displayunit and detects a user who is present near each side edge.(103)The information processing apparatus according to (101), wherein thecomputation unit sets a user occupying area for each of the detectedusers and a common area which is commonly used by the users in thescreen of the display unit in accordance with user arrangement detectedby the user detection unit.(104)The information processing apparatus according to (103), wherein thecomputation unit displays one or more operation target objects astargets to be operated by the user on the screen of the display unit.(105)The information processing apparatus according to (104), wherein thecomputation unit optimizes the operation target object in a useroccupying area.(106)The information processing apparatus according to (104), wherein thecomputation unit performs rotation processing such that the operationtarget object in a user occupying area is in a direction in which theoperation target object faces a corresponding user.(107)The information processing apparatus according to (104), wherein thecomputation unit performs rotation processing such that an operationtarget object which has been displaced from the common area or anotheruser occupying area to the user occupying area is in a direction inwhich the operation target object faces a corresponding user.(108)The information processing apparatus according to (107), wherein thecomputation unit controls a rotation direction in performing therotation processing on the operation target object in accordance with aposition, at which the user has performed an operation, with respect toa position of the center of gravity of the operation target object whenthe user drags and displaces the operation target object between areas.(109)The information processing apparatus according to (103), wherein thecomputation unit displays a detection indicator representing that a userhas been newly detected when the user detection unit sets a useroccupying area for the newly detected user in the screen of the displayunit.(110)The information processing apparatus according to (104), furtherincluding a data exchange unit which exchange data with a terminal ownedby a user.(111)The information processing apparatus according to (110), wherein thedata exchange unit performs data exchange processing with the terminalowned by the user who has been detected by the user detection unit, andwherein the computation unit causes an operation target objectcorresponding to data received from the terminal owned by the user toappear in a corresponding occupying area.(112)The information processing apparatus according to (104), wherein thecomputation unit copies an operation target object to a user occupyingarea as a displacement destination or divides the operation targetobject for the user occupying area as the displacement destination inresponse to displacement of the operation target object between useroccupying areas for users.(113)The information processing apparatus according to (112), wherein thecomputation unit displays a copy of the operation target object createdas separate data in the user occupying area as the displacementdestination.(114)The information processing apparatus according to (112), wherein thecomputation unit displays a copy of the operation target object asanother window of an application, which can be commonly operated by theusers, in the user occupying area as the displacement destination.(115)An information processing method including: a user detection step inwhich a user who is present in the periphery is detected; a computationstep in which processing is performed on a displayed operation targetobject in response to detection of the user in the user detection step.(116)A computer program described in a computer readable format so as tocause a computer to function as: a display unit; a user detection unitwhich detects a user who is present in the periphery of the displayunit; and a computation unit which performs processing on an operationtarget object displayed on the display unit in response to detection ofthe user by the user detection unit.(201)An information processing apparatus including: a display unit; a userposition detection unit which detects a user position with respect tothe display unit; a user state detection unit which detects a user statewith respect to a display screen of the display unit; and a computationunit which controls a GUI to be displayed on the display unit inaccordance with the user position detected by the user positiondetection unit and the user state detected by the user state detectionunit.(202)The information processing apparatus according to (201), wherein thecomputation unit controls frameworks or information density of one ormore operation target objects as targets to be operated by the user,which are displayed on the screen of the display unit, in accordancewith the user position and the user state.(203)The information processing apparatus according to (201), wherein thecomputation unit controls a framework of an operation target object tobe displayed on the screen in accordance with whether or not the user isin a state of viewing the screen of the display unit.(204)The information processing apparatus according to (201), wherein thecomputation unit controls information density of an operation targetobject to be displayed on the screen of the display unit in accordancewith the user position.(205)The information processing apparatus according to (201), wherein thecomputation unit controls selection of an operation target object to bedisplayed on the screen of the display unit in accordance with whetheror not the user is at a position where personal authentication can beperformed.(206)The information processing apparatus according to (201), furtherincluding one or more input means for allowing the user to operate anoperation target object being displayed on the screen of the displayunit, wherein the computation unit controls a framework of the operationtarget object to be displayed on the screen in accordance with whetheror not the user is in a state of operating the operation target objectwith the input means.(207)An information processing apparatus including: a display unit, one ormore input means for allowing a user to operate an operation targetobject being displayed on a screen of the display unit; a user positiondetection unit which detects a user position with respect to the displayunit; a user state detection unit which detects a user state withrespect to a display screen of the display unit; and a computation unitwhich optimizes the input means in accordance with the user positiondetected by the user position detection unit and the user state detectedby the user state detection unit.(208)The information processing apparatus according to (207), wherein thecomputation unit controls optimization of the input means in accordancewith whether or not the user is in a state of viewing the screen of thedisplay unit.(209)The information processing apparatus according to (207), wherein thecomputation unit optimizes the input means in accordance with the userposition detected by the user position detection unit in a state inwhich the user is viewing the screen of the display unit.(210)An information processing apparatus including: a display unit; a userposition detection unit which detects a user position with respect tothe display unit; and a computation unit which controls a plurality ofdistance detecting schemes for detecting a distance from the screen ofthe display unit to the user and shifting of the distance detectingschemes in accordance with the user position detected by the userposition detection unit.(211)The information processing apparatus according to (210), wherein thecomputation unit constantly turns on a function of a distance detectingscheme for detecting a distance to a user who is at a distant place.(212)The information processing apparatus according to (210), wherein thecomputation unit turns on a function of a distance detecting scheme fordetecting a distance to a user who is at a nearby place and alsoperforming recognition processing only within a distance range in whichsufficient recognition accuracy can be obtained.(213)An information processing method including: a user position detectionstep in which a user position with respect to a display screen isdetected; a user state detection step in which a user state with respectto the display screen is detected, and a computation step in which a GUIto be displayed on the display screen is controlled based on the userposition detected in the user position detection step and the user statedetected in the user state detection step.(214)An information processing method including: a user position detectionstep in which a user position with respect to a display screen isdetected; a used state detection step in which a used state with respectto the display screen is detected; and a computation step in which oneor more input means for allowing the user to operate an operation targetobject being displayed on the display screen are optimized in accordancewith the user position detected in the user position detection step andthe user state detected in the user state detection step.(215)An information processing method including: a user position detectionstep in which a user position with respect to a display screen isdetected; and a computation step in which shifting of a plurality ofdistance detecting schemes for detecting a distance from the displayscreen to the user is controlled in accordance with the user positiondetected in the user position detection step.(216)A computer program described in a computer readable format so as tocause a computer to function as: a display unit; a user positiondetection unit which detects a user position with respect to the displayunit; a user state detection unit which detects a user state withrespect to a display screen of the display unit; and a computation unitwhich controls a GUI to be displayed on the display unit in accordancewith the user position detected by the user position detection unit andthe user state detected by the user state detection unit.(217)A computer program described in a computer readable format so as tocause a computer to function as: a display unit, one or more input meansfor allowing a user to operate an operation target object beingdisplayed on a screen of the display unit; a user position detectionunit which detects a user position with respect to the display unit; auser state detection unit which detects a user state with respect to adisplay screen of the display unit; and a computation unit whichoptimizes the input means in accordance with the user position detectedby the user position detection unit and the user state detected by theuser state detection unit.(218)A computer program described in a computer readable format so as tocause a computer to function as: a display unit; a user positiondetection unit which detects a user position with respect to the displayunit; and a computation unit which controls a plurality of distancedetecting schemes for detecting a distance from the screen of thedisplay unit to the user and shifting of the distance detecting schemesin accordance with the user position detected by the user positiondetection unit.(301)An information processing apparatus including: a display unit; an objectimage obtaining unit which obtains an object image to be displayed on ascreen of the display unit; a real size obtaining unit which obtainsinformation relating to a real size of the object to be displayed on thescreen of the display unit; and a computation unit which processes theobject image based on the real size of the object obtained by the realsize obtaining unit.(302)The information processing apparatus according to (301), furtherincluding a display performance obtaining unit which obtains informationon display performances including a screen size and resolution of thedisplay unit, wherein the computation unit processes the object image tobe displayed at the real size on the screen of the display unit based onthe real size of the object obtained by the real size obtaining unit andthe display performances obtained by the display performance obtainingunit.(303)The information processing apparatus according to (301), wherein insimultaneously displaying a plurality of object images obtained by theobject image obtaining unit on the screen of the display unit, thecomputation unit processes the plurality of object image such thatmutual magnitude relationships among the plurality of object images arecorrectly displayed.(304)The information processing apparatus according to (301), furtherincluding: a camera unit; and a real size estimation unit whichestimates a real size of an object included in an image captured by thecamera unit.(305)The information processing apparatus according to (301), furtherincluding: a camera unit, an image recognition unit which recognizes auser face included in an image captured by the camera unit and obtainsface data; a distance detection unit which detects a distance to theuser; and a real size estimation unit which estimates a real size of theuser face based on the face data of the user and a distance to the user.(306)An information processing method including: an object image obtainingstep in which an object image to be displayed on a screen is obtained; areal size obtaining step in which information relating to a real size ofthe object to be displayed on the screen is obtained; and a computationstep in which the object image is processed based on the real size ofthe object obtained in the real size obtaining step.(307)A computer program described in a computer readable format so as tocause a computer to function as: a display unit; an object imageobtaining unit which obtains an object image to be displayed on a screenof the display unit; a real size obtaining unit which obtainsinformation relating to a real size of the object to be displayed on thescreen of the display unit; and a computation unit which processes theobject image based on the real size of the object obtained by the realsize obtaining unit.(401)An information processing apparatus including: a camera unit; a displayunit; and a computation unit which normalizes a user image captured bythe camera unit in displaying the image on a screen of the display unit.(402)The information processing apparatus according to (401), furtherincluding: an object image obtaining unit which obtains an object imageto be displayed on the screen of the display unit; and a parallel orsuperimposed pattern obtaining unit which obtains a parallel orsuperimposed pattern in which the user image and the object image arearranged in parallel or superimposed on the screen of the display unit,wherein the computation unit performs normalization such that amagnitude relationship between the user image and the object iscorrectly maintained and arranged in parallel or superimposes the userimage and the object after the normalization based on the obtainedparallel or superimposed pattern.(403)The information processing apparatus according to (402), wherein thecomputation unit controls the camera unit in order to normalize the userimage captured by the camera unit.(404)The information processing apparatus according to (401), furtherincluding: a user face data obtaining unit which obtains user face datacaptured by the camera unit; and an intra-object face data obtainingunit which obtains face data in an object to be displayed on the screenof the display unit, wherein the computation unit performs normalizationsuch that a magnitude relationship and positions of the user face dataand the face data in the object are correctly maintained.(405)The information processing apparatus according to (404), wherein thecomputation unit controls the camera unit in order to normalize the userimage captured by the camera unit.(406)An information processing method including: an object image obtainingstep in which an object image to be displayed on the screen is obtained;a parallel or superimposed pattern obtaining step in which a parallel orsuperimposed pattern, in which a user image captured by the camera unitand the object image are arranged in parallel or superimposed on thescreen of the display unit, is obtained; a normalization step in whichnormalization is performed such that a magnitude relationship and aposition of the user image and the object are correctly maintained; andan image retouching step in which the user image and the object afterthe normalization are arranged in parallel or superimposed based on theobtained parallel or superimposed pattern.(407)An information processing method including: a user face data obtainingstep in which user face data captured by the camera unit is obtained; anintra-object face data obtaining step in which face data in an object tobe displayed on the screen is obtained; and a normalization step inwhich normalization is performed such that a magnitude relationship andpositions of the user face data and the face data in the object arecorrectly maintained.(408)A computer program described in a computer readable format so as tocause a computer to function as: a camera unit; a display unit; and acomputation unit which normalizes a user image captured by the cameraunit in displaying the user image on a screen of the display unit.(501)An information processing apparatus including: a display unit whichdisplays movie content on a screen; a rotation angle detection unitwhich detects a rotation angle of the screen; a display mode selectingunit which selects a movie content display mode at an arbitrary rotationangle of the screen in a shifting process thereof; and an imageretouching unit which performs image retouch based on the display modeselected by the display mode selecting unit such that the movie contentsuits the screen inclined at the rotation angle detected by the rotationangle detection unit.(502)The information processing apparatus according to (501), wherein thedisplay mode selecting unit selects one of a plurality of display modesincluding three display modes, namely a display mode for allowing themovie content to be completely displayed on the screen at an arbitraryrotation angle, a display mode for maximizing focused content in themovie content at each rotation angle, and a display mode for rotatingthe movie content so as not to generate an invalid area.(503)The information processing apparatus according to (501), wherein thedisplay mode selecting unit selects a display mode at an arbitraryrotation angle of the screen in a shifting process thereof based onattribute information included in the movie content.(504)The information processing apparatus according to (501), wherein thedisplay mode selecting unit selects for protected movie content thedisplay mode for allowing the movie content to be completely displayedon the screen at an arbitrary rotation angle.(505)An information processing method including: a rotation angle detectionstep in which a rotation angle of a screen for displaying movie contentis detected; a display mode selecting step in which a movie contentdisplay mode is selected at an arbitrary rotation angle of the screen inthe shifting process thereof; and an image retouching step in whichimage retouching is performed based on the display mode selected by thedisplay mode selecting step such that the movie content suits the screeninclined at the rotation angle detected in the rotation angle detectionstep.(506)A computer program described in a computer readable format so as tocause a computer to function as: a display unit which display moviecontent on a screen; a rotation angle detection unit which detects arotation angle of the screen; a display mode selecting unit whichselects a movie content display mode at an arbitrary rotation angle ofthe screen in a shifting process thereof; and an image retouch unitwhich performs image retouching based on the display mode selected bythe display mode selecting unit such that the movie content suits thescreen inclined at the rotation angle detected by the rotation angledetection unit.

As described above, the technique disclosed in this specification wasdescribed in detail with reference to specific embodiments. However, itis obvious that those skilled in the art can make modifications andreplacements of the embodiments without departing from a scope of thetechnique disclosed in the specification.

In the present specification, although the description was mainly givenof embodiments on the assumption of a television receiver provided witha large screen as the information processing apparatus 100 to which thetechnique disclosed in this specification, the scope of the techniquedisclosed in the specification is not limited thereto. It is possible tosimilarly apply the technique disclosed in this specification to aninformation processing apparatus other than the television receiver suchas a personal computer and a tablet terminal and an informationprocessing apparatus with a screen which is not large.

In short, the technique disclosed in this specification was described asillustrative examples, and the contents described in this specificationshould not be exclusively understood. Claims should be taken intoaccount in order to determine the scope of the technique disclosed inthis specification.

REFERENCE SIGNS LIST

-   -   100 INFORMATION PROCESSING APPARATUS    -   110 INPUT INTERFACE UNIT    -   120 COMPUTATION UNIT    -   130 OUTPUT INTERFACE UNIT    -   140 STORAGE UNIT    -   150 COMMUNICATION UNIT    -   160 POWER UNIT    -   170 TELEVISION TUNER UNIT    -   180 ROTATION AND ATTACHMENT MECHANISM UNIT    -   501 REMOTE CONTROL RECEIVING UNIT    -   502 SIGNAL ANALYSIS UNIT    -   503 CAMERA UNIT    -   504 IMAGE RECOGNITION UNIT    -   505 MICROPHONE UNIT    -   506 AUDIO RECOGNITION UNIT    -   507 DISTANCE SENSOR    -   508 SIGNAL ANALYSIS UNIT    -   509 TOUCH DETECTION UNIT    -   510 SIGNAL ANALYSIS UNIT    -   511 PROXIMITY SENSOR    -   512 SIGNAL ANALYSIS UNIT    -   513 VERY-SHORT-RANGE COMMUNICATION UNIT    -   514 SIGNAL ANALYSIS UNIT    -   515 TRIAXIAL SENSOR UNIT    -   516 GPS RECEIVING UNIT    -   517 SIGNAL ANALYSIS UNIT    -   520 INPUT INTERFACE UNIFYING UNIT    -   601 CONTENT DISPLAY UNIT    -   602 GUI DISPLAY UNIT    -   603 DISPLAY UNIT    -   604 SPEAKER UNIT    -   605 ILLUMINATION DISPLAY UNIT    -   606 ILLUMINATION UNIT    -   710 MONITOR AREA DIVIDING UNIT    -   711 DEVICE DATABASE    -   712 AREA PATTERN DATABASE    -   720 OBJECT OPTIMAL PROCESSING UNIT    -   721 OPTIMAL PROCESSING ALGORITHM    -   730 EQUIPMENT COORDINATION DATA EXCHANGING UNIT    -   731 EXCHANGE PROCESSING ALGORITHM    -   2310 DISPLAY GUI OPTIMIZATION UNIT    -   2320 INPUT MEANS OPTIMIZATION UNIT    -   2330 DISTANCE DETECTING SCHEME SHIFTING UNIT    -   3210 REAL SIZE DISPLAY UNIT    -   3220 REAL SIZE ESTIMATION UNIT    -   3230 REAL SIZE EXPANSION UNIT    -   4110 INTER-IMAGE NORMALIZATION PROCESSING UNIT    -   4120 FACE NORMALIZATION PROCESSING UNIT    -   4130 REAL SIZE EXPANSION UNIT    -   4710 DISPLAY MODE SELECTING UNIT    -   4720 ROTATION POSITION INPUT UNIT    -   4730 IMAGE RETOUCH UNIT

1. An information processing apparatus comprising: a control unit tocontrol processing on an operation object, wherein an operation right ofthe operation object is given to a user, displayed on a display unitbased on detection of a user from at least one information signal inputto the apparatus from outside of the apparatus.
 2. The apparatus ofclaim 1, wherein the control unit sets a user occupying area in a screenof the display unit for each user detected from the at least oneinformation signal.
 3. The apparatus of claim 1, wherein the controlunit sets a common area in a screen of the display unit usable by eachuser detected from the at least one information signal.
 4. The apparatusof claim 1, wherein the control unit sets (i) a user occupying area in ascreen of the display unit for each user detected from the at least oneinformation signal, and (ii) all area of the screen other than the useroccupying area of the each user detected as a common area usable by theeach user detected.
 5. The apparatus of claim 1 further comprising: atleast one proximity sensor disposed at each of four side edge portionsof a screen of the display unit and at which the at least oneinformation signal is input.
 6. The apparatus of claim 1, wherein thecontrol unit automatically rotates the operation object, when present ina user occupying area set for the user in a screen of the display unit,to face the user.
 7. The apparatus of claim 6, wherein the operationobject is automatically rotated in accordance with a predeterminedtiming.
 8. The apparatus of claim 1, wherein the control unit rotatesthe operation object in accordance with a position, at which the userhas performed an operation with respect to a position of a center ofgravity of the operation object, when the user displaces the operationobject between set first and second areas in a screen of the displayunit.
 9. The apparatus of claim 8, wherein the user displaces theoperation object by dragging or throwing.
 10. The apparatus of claim 1further comprising: a data exchange unit to exchange data with aterminal owned by the user.
 11. The apparatus of claim 10, wherein thecontrol unit displays the operation object on a screen of the displayunit, based on the data exchanged with the terminal.
 12. The apparatusof claim 1, wherein the control unit controls at least one of copying ordividing of the operation object, in response to an operation fordisplacing the operation object.
 13. The apparatus of claim 1 furthercomprising: an input interface unit to which the at least oneinformation signal from the outside is input.
 14. The apparatus of claim1, wherein the control unit controls the processing on the operationobject based on at least one of a screen touch detection result or datareceived from a terminal owned by the user.
 15. The apparatus of claim1, wherein the control unit controls the processing on the operationobject by selecting one of a plurality of input units of the apparatus.16. The apparatus of claim 15, wherein the selecting is in accordancewith position of or distance to the user.
 17. The apparatus of claim 1,wherein the apparatus includes the display unit.
 18. The apparatus ofclaim 1, wherein the processing on the operation object is based on astate of the user.
 19. An information processing method comprising:controlling, by a processor, of processing on an operation object,wherein an operation right of the operation object is given to a user,displayed on a display unit based on detection of a user from at leastone information signal input to an information processing apparatus fromoutside of the apparatus.
 20. A non-transitory recording medium recordedwith a program executable by a computer, the program comprising:controlling processing on an operation object, wherein an operationright of the operation object is given to a user, displayed on a displayunit based on detection of a user from at least one information signalinput to an information processing apparatus from outside of theapparatus.